-- Logs begin at Thu 2022-10-20 12:57:22 UTC, end at Thu 2022-10-20 13:53:44 UTC. -- Oct 20 13:21:54.671864 dv-cinder-862136-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 20 13:21:54.679365 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 20 13:21:54.683203 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Oct 20 13:21:54 2022] *** Oct 20 13:21:54.683203 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 20 13:21:54.683203 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 20 13:21:54.683203 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: nodename: dv-cinder-862136-1-iscsi Oct 20 13:21:54.683203 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: machine: x86_64 Oct 20 13:21:54.683203 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: clock source: unix Oct 20 13:21:54.683203 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: pcre jit disabled Oct 20 13:21:54.683203 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: detected number of CPU cores: 4 Oct 20 13:21:54.683203 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: current working directory: / Oct 20 13:21:54.683203 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: detected binary path: /usr/bin/uwsgi-core Oct 20 13:21:54.683203 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: your processes number limit is 47680 Oct 20 13:21:54.683203 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: your memory page size is 4096 bytes Oct 20 13:21:54.683203 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: detected max file descriptor number: 2048 Oct 20 13:21:54.683203 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: lock engine: pthread robust mutexes Oct 20 13:21:54.683203 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: thunder lock: enabled Oct 20 13:21:54.683203 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 20 13:21:54.683203 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 20 13:21:54.722663 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: Python main interpreter initialized at 0x561cbcb029c0 Oct 20 13:21:54.722663 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: python threads support enabled Oct 20 13:21:54.722663 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: your server socket listen backlog is limited to 100 connections Oct 20 13:21:54.722663 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: your mercy for graceful operations on workers is 90 seconds Oct 20 13:21:54.723040 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: mapped 403077 bytes (393 KB) for 2 cores Oct 20 13:21:54.723118 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: *** Operational MODE: preforking *** Oct 20 13:21:54.723369 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: *** uWSGI is running in multiple interpreter mode *** Oct 20 13:21:54.723369 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: spawned uWSGI master process (pid: 81460) Oct 20 13:21:54.723575 dv-cinder-862136-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 20 13:21:54.724895 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: spawned uWSGI worker 1 (pid: 81474, cores: 1) Oct 20 13:21:54.724895 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: spawned uWSGI worker 2 (pid: 81475, cores: 1) Oct 20 13:21:54.724895 dv-cinder-862136-1-iscsi devstack@keystone.service[81460]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 20 13:21:59.003730 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 20 13:21:59.003730 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: from cryptography.utils import int_from_bytes Oct 20 13:21:59.004516 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 20 13:21:59.004516 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: from cryptography.utils import int_from_bytes Oct 20 13:21:59.004636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 20 13:21:59.004636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: from cryptography.utils import int_from_bytes Oct 20 13:21:59.004636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 20 13:21:59.004636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: from cryptography.utils import int_from_bytes Oct 20 13:21:59.054851 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 20 13:21:59.055029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 20 13:21:59.055221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] command line args: [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 20 13:21:59.055313 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 20 13:21:59.055482 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 20 13:21:59.055565 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 20 13:21:59.055665 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ================================================================================ {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 20 13:21:59.055741 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] command line args: [] {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 20 13:21:59.055906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] admin_token = **** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.055995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 20 13:21:59.056097 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ================================================================================ {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 20 13:21:59.056176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] config_dir = [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.056330 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.056415 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] admin_token = **** {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.056515 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] config_source = [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.056600 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] config_dir = [] {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.056699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] debug = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.056776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.056931 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.057184 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] config_source = [] {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.057184 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] debug = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.057304 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.057362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.057534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.057613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.057707 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.057777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.057961 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.058086 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.058190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] list_limit = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.058291 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.058386 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] log_config_append = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.058479 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] list_limit = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.058578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.058681 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] log_config_append = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.058797 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] log_dir = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.058965 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.059140 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] log_file = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.059338 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] log_dir = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.059482 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.059482 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.059664 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.059791 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.059958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.060136 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.060428 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.060682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.060947 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.061187 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.061426 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.061663 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.061922 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.062174 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.062454 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.062712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] log_file = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.062953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.063137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] publish_errors = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.063372 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.063447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.063606 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.063718 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.063800 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.063991 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.064075 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.064247 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.064330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.064444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.064530 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] standard_threads = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.064636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.064719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.064925 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.065064 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.065151 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.065230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.065307 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] use_journal = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.065419 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.065503 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] use_json = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.065621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] use_stderr = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.065709 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.065789 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] use_syslog = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.065886 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.066019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.066126 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.066277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.066378 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.066518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.066629 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.066780 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.066908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.067022 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.067273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.067398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.067516 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.067651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] publish_errors = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.067765 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.067849 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.068002 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.068079 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.068186 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.068270 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.068434 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.068511 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.068669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.068751 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.069529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.069701 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] standard_threads = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.069777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.069891 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.070025 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.070138 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.070222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.070302 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.070473 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.070557 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] use_journal = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.070648 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] use_json = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.070771 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.070852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] use_stderr = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.070953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.071039 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] use_syslog = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.071205 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 20 13:21:59.071292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.071450 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.071532 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.071686 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.071763 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.071944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.072020 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.072182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.072257 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.072431 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.072509 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.072669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.072752 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.072924 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.073005 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.073161 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.073247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.073394 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.073473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.073629 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.073707 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.073857 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.073929 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.074093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.074170 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.074334 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.074419 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.074567 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.074643 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.074812 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.074889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.075056 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.075144 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.075292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.075374 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.075544 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.075619 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.075774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.075859 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.076012 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.076096 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.076249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.076331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.076481 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.076566 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.076715 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.076793 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.076975 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.077055 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.077211 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.077298 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.077448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.077526 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.077677 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.077752 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.077909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.077995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.078143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.078226 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.078372 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.078448 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.078608 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.078693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.078837 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.078913 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.079066 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.079143 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.079303 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.079457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.079531 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.079607 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.079767 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.079845 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.080006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.080091 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.080233 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.080316 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.080469 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.080547 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.080699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.080802 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.080947 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.081030 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.081180 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.081259 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.081410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.081523 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.081640 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.081744 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.081875 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.081980 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.082121 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.082231 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.082351 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.082437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.082584 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.082667 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.082801 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.082891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.083032 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.083153 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.083265 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.083345 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.083509 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.083595 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.083740 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.083823 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.083975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.084102 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.084208 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.084294 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.084465 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.084618 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.084724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.084916 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.085000 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.085146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.085234 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.085388 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.085466 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.085630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.085710 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.085865 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.085954 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.086093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.086172 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.086341 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.086422 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.086573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.086699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.086803 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.086882 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.087038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.087122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.087269 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.087354 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.087529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.087655 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.087762 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.087838 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.088007 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.088094 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.088246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.088329 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.088477 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.088554 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.088709 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.088795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.088965 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.089047 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.089212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.089291 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.089449 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.089542 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.089685 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.089764 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.089921 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.089999 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.090154 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.090240 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.090394 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.090473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.090627 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.090705 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.090859 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.091053 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.091132 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.091202 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.091321 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.091398 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.091554 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.091642 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.091812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.091899 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.092051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.092130 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.092298 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.092381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.092533 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] auth.external = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.092613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.092765 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.092843 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.093028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.093106 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.093267 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.093348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.093504 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.093582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] auth.password = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.093740 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.093821 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] auth.token = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.093970 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.094050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.094201 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.094278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.094431 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.094535 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.094667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.094748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.094898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.094979 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.095136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.095222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.095371 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.095448 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.095609 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] credential.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.095690 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.095843 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.095920 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.096095 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.096173 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.096342 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.096420 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.096581 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.096659 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.096811 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] auth.external = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.096907 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.097067 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.097154 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.097313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.097390 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.097569 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.097649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.097797 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] auth.password = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.097876 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.098027 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] auth.token = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.098106 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.098267 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.098356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.098502 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.098588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.098747 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.098826 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.098977 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.099055 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.099221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.099302 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.099468 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.099547 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] federation.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.099701 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.099780 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.099936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] credential.caching = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.100021 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.100209 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.100301 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.100444 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.100525 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.100688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.100768 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.100948 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.101031 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.101191 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.101269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.101423 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.101506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.101666 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.101745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.101909 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.101986 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.102143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.102222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.102384 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.102462 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.102633 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.102720 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.102881 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.102966 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.103117 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.103196 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.103354 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.103442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.103603 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.103682 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.103832 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] federation.caching = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.103916 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.104069 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.104158 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.104314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.104394 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.104557 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.104637 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.104792 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.104992 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.105072 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.105144 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.105280 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.105359 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.105526 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.105613 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.105763 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.105843 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.106009 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.106087 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.106243 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.106329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.106491 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.106571 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.106724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] identity.caching = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.106803 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.106969 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.107056 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.107214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.107293 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.107443 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.107529 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.107674 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.107761 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.107905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.107983 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.108135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.108219 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.108372 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.108459 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.108611 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.108689 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.108884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.109017 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.109115 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.109259 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.109341 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.109514 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.109593 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.109766 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.109845 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.110007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.110088 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.110258 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.110338 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.110508 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.110586 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.110741 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.110821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.110984 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.111060 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.111218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.111297 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.111460 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.111539 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.111691 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.111771 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.111929 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.112008 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.112160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.112240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.112394 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.112472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.112633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.112713 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.112888 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.112971 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.113132 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.113263 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.113368 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.113500 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.113619 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.113750 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.113871 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.113999 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.114110 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.114235 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.114349 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.114481 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.114587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.114734 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.114813 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.114966 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.115047 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.115207 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.115286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.115439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.115518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.116208 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.116303 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.116506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.116648 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.116752 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.116964 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.117060 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.117212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.117329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.117454 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.117532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.117688 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.117774 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.117926 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.118006 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.118158 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.118237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.118388 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.118474 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.118639 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.118718 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.118882 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.118960 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.119115 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.119207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.119351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.119430 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.119579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.119658 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.119806 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.user = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.119891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.120045 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.120124 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.120282 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.120359 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.120519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.120607 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.120756 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.120835 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.121024 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.121104 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.121272 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.121416 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.121495 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.121565 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.121718 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.121799 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.121952 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.122040 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.122186 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.122265 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.122421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.122499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.122650 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.122736 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.122887 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.122965 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.123126 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.123204 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.123368 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.123455 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.123602 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.123680 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.123840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.123919 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.124066 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.124152 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] resource.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.124314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.124393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.124548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.124627 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.124784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.124890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.125061 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.125179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.125280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.125359 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.125524 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.125616 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.125768 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.125846 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.126001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.126078 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.126240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.126326 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.126467 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.126545 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] role.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.126710 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.126788 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] role.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.126941 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.127027 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.127172 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.127249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.127408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.127487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.127653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.127733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.127893 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.127972 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.128124 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.128203 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.128352 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.128428 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.128585 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] resource.caching = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.128663 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.128832 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.128929 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.129100 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.129181 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.129337 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.129415 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.129584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.129662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.129833 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.129913 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.130065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.130143 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.130309 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.130461 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.130549 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.130703 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.130782 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.130955 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.131034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] role.caching = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.131207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.131294 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] role.driver = sql {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.132010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.132133 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.132270 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.132412 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.132511 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.132662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.132769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.132937 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.133071 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.133187 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.133292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.133433 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.133515 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.133679 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.133759 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.133926 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.134005 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.134173 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.134253 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.134403 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.134482 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.134648 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.134726 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.134898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.134979 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.135138 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.135216 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.135381 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.135473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.135610 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] token.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.135762 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.135841 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.136001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.136079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.136253 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.136334 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.136503 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.136582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.136752 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.137109 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.137296 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.137378 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.137533 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.137612 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.137784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.137861 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.138034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.138113 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.138271 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.138349 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.138502 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.138579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.138735 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.138815 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.138966 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.139045 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.139217 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.139286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.139459 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.139551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.139704 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.139784 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.139936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.140013 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.140170 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.140257 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.connection = **** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.140401 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.140479 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.140636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] token.caching = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.140714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.140930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.141028 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.141140 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.141220 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.141375 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.141457 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.141612 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.141699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.141851 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.141930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.142085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.142163 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.142330 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.142416 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.142570 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.142648 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.142816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.142893 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.143051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.143137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.143288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.143367 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.143520 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.143599 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.143765 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.143852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.143995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.144073 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.144239 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.144318 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.144477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.144565 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.144639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81475) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 20 13:21:59.144739 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.145031 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] database.connection = **** {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.145278 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.145525 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.145764 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.146003 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.146238 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.146473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.146707 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.146973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.147211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.147450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.147695 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.147931 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.148178 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.148415 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.148661 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.148982 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.149226 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.149462 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 20 13:21:59.149630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81474) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 20 13:21:59.174450 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.174679 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.174903 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.175202 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.175436 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.175660 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.183899 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.184433 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.211972 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.212085 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.212199 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.212282 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.265607 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.265804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.313934 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.314163 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.314443 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.314685 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.469501 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.469691 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.469805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.469885 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.469960 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.470060 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.470140 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.470214 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.470340 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.470424 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.470575 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.470662 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.495273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.495506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.495715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.495835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.495960 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.496067 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.496169 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.496270 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.496370 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.496492 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.496597 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.496698 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.496797 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.496989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.508027 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.508385 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.521007 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.521187 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.521414 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.521521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.521637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.521729 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.521856 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.522476 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.522624 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.522725 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.522790 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81474) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.522907 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.523134 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.523353 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81475) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 20 13:21:59.539983 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.540183 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 20 13:21:59.540526 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.540642 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.540726 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 20 13:21:59.540854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 20 13:21:59.541011 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.541221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.541303 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.541423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 20 13:21:59.541518 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.541700 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.541782 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.541964 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.542225 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.542484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.551415 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.551694 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.551963 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.552205 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.552288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.552490 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.552764 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.553076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.559264 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81474) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.560324 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.562213 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81474) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.562498 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.562781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.563062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.563281 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.563364 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.563570 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.563654 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.563845 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.563929 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.564119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.564209 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.564387 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.564468 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.564665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.564746 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.564954 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.565039 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.565235 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.565511 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.565787 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.566060 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.588626 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.589499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.590143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81474) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.590427 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.590982 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.591270 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.595124 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81474) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.595411 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.595695 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.595992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.596278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.596568 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.602839 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81474) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.603115 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.603784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.604076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.608732 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81474) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.609060 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.609348 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.609635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.609803 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.609908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.610100 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.610183 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.610392 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.610474 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.610685 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.610768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.610968 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.611249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.611527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.611803 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.626936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.627214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.627584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.627867 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.631107 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81474) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.631870 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.634567 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.634866 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.635171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.635471 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.639362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81474) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.640006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.642559 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81474) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.643212 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.645814 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.646113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.646400 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.646523 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.646687 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.646828 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.647514 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.647641 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.647801 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.648015 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.648192 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.648470 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.664479 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81474) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.664762 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.664965 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.665050 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.665258 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.665355 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.665529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.665794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.672154 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.672709 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.673728 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.674285 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.675257 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.675533 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.675830 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.676110 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.678509 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81474) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.678804 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.678964 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.679088 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.679250 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.679370 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.679536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.679769 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.679860 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.680212 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.690950 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81474) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.691237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.691442 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.691525 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.691732 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.691827 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.692016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.692165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.692299 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.692447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.692683 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.692766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.692990 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.693284 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.708205 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81474) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.708829 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.711673 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81474) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.712454 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.714618 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.715413 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.716074 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.716997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.717575 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81474) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.717858 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.718267 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.718488 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.718773 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.719182 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.724632 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.726491 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.726721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81474) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.727991 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.729638 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.730036 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.730322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.730600 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.730949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.731351 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.731643 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.731922 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.738471 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81474) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.738764 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.739048 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.739809 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.740105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.740393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.746130 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81474) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.746415 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.746696 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.746973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.747245 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.747521 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.747705 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81475) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 20 13:21:59.747796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.747992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.748075 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.748277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.748358 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.748565 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.748658 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.748842 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.748946 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.749130 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.749203 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.749406 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.749498 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.749685 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.749767 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81474) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.749956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.750250 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.750524 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.750789 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.751065 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.751379 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81475) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 20 13:21:59.784519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x561cbcb029c0 pid: 81474 (default app) Oct 20 13:21:59.786505 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x561cbcb029c0 pid: 81475 (default app) Oct 20 13:21:59.787806 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c766719-7413-45a7-b7da-d7c5e9218196 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:21:59.788094 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c766719-7413-45a7-b7da-d7c5e9218196 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:21:59.788365 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c766719-7413-45a7-b7da-d7c5e9218196 None None] PATH_INFO: `/v3` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:21:59.792511 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Thu Oct 20 13:21:59 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:06.552413 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e91adc-0d8c-4c12-a5ab-8026fcec15c8 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:06.552893 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e91adc-0d8c-4c12-a5ab-8026fcec15c8 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:06.553000 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e91adc-0d8c-4c12-a5ab-8026fcec15c8 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:06.557411 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1/2] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:22:06.562723 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b3923-0d0e-4da2-a75a-5aa28ed0e745 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:06.563185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b3923-0d0e-4da2-a75a-5aa28ed0e745 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:06.563323 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b3923-0d0e-4da2-a75a-5aa28ed0e745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:06.633901 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: INFO dbcounter [None req-823b3923-0d0e-4da2-a75a-5aa28ed0e745 None None] Registered counter for database keystone Oct 20 13:22:06.652743 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG oslo_db.sqlalchemy.engines [None req-823b3923-0d0e-4da2-a75a-5aa28ed0e745 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81474) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 20 13:22:06.658889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writer thread running {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 20 13:22:06.836454 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG passlib.handlers.bcrypt [None req-823b3923-0d0e-4da2-a75a-5aa28ed0e745 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81474) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 20 13:22:06.836846 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG passlib.handlers.bcrypt [None req-823b3923-0d0e-4da2-a75a-5aa28ed0e745 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81474) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 20 13:22:06.894773 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-823b3923-0d0e-4da2-a75a-5aa28ed0e745 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:06.896387 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-823b3923-0d0e-4da2-a75a-5aa28ed0e745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:07.021828 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 2/3] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 20 13:22:07.494778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e91adc-0d8c-4c12-a5ab-8026fcec15c8 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:07.566433 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: INFO dbcounter [None req-26e91adc-0d8c-4c12-a5ab-8026fcec15c8 None None] Registered counter for database keystone Oct 20 13:22:07.588655 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG oslo_db.sqlalchemy.engines [None req-26e91adc-0d8c-4c12-a5ab-8026fcec15c8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81475) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 20 13:22:07.595148 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writer thread running {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 20 13:22:07.717535 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e91adc-0d8c-4c12-a5ab-8026fcec15c8 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:07.721658 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1147b01-f581-4ec8-a283-c869894c55c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:07.722283 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1147b01-f581-4ec8-a283-c869894c55c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:07.722568 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1147b01-f581-4ec8-a283-c869894c55c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:07.722842 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1147b01-f581-4ec8-a283-c869894c55c2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:07.731463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1147b01-f581-4ec8-a283-c869894c55c2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:07.784433 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b1147b01-f581-4ec8-a283-c869894c55c2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 20 13:22:07.784737 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1147b01-f581-4ec8-a283-c869894c55c2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:07.794623 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 2/4] 10.222.0.23 () {58 vars in 1324 bytes} [Thu Oct 20 13:22:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 13:22:07.800578 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b3923-0d0e-4da2-a75a-5aa28ed0e745 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:07.817577 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b3923-0d0e-4da2-a75a-5aa28ed0e745 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:07.823008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73054c31-8992-4e0d-9f78-c4714c82fb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:07.824371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73054c31-8992-4e0d-9f78-c4714c82fb93 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:07.824655 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73054c31-8992-4e0d-9f78-c4714c82fb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:07.824958 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73054c31-8992-4e0d-9f78-c4714c82fb93 admin admin] PATH_INFO: `/v3/projects/c7d49958a3a448d0bed98efad34bc0a2` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:07.826552 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73054c31-8992-4e0d-9f78-c4714c82fb93 admin admin] RBAC: Authorizing `identity:get_project(project_id=c7d49958a3a448d0bed98efad34bc0a2)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:07.880243 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: WARNING keystone.common.rbac_enforcer.enforcer [None req-73054c31-8992-4e0d-9f78-c4714c82fb93 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 20 13:22:07.880595 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73054c31-8992-4e0d-9f78-c4714c82fb93 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:07.886127 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 3/5] 10.222.0.23 () {56 vars in 1207 bytes} [Thu Oct 20 13:22:07 2022] GET /identity/v3/projects/c7d49958a3a448d0bed98efad34bc0a2 => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:22:10.044095 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2764d-5907-45d0-82a8-18ac7286a4b8 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:10.044624 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2764d-5907-45d0-82a8-18ac7286a4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:10.044624 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2764d-5907-45d0-82a8-18ac7286a4b8 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:10.048781 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 3/6] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:10.054410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44132df8-9443-4911-93d9-cc6db22b09a1 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:10.054808 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44132df8-9443-4911-93d9-cc6db22b09a1 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:10.055075 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44132df8-9443-4911-93d9-cc6db22b09a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:10.075711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-44132df8-9443-4911-93d9-cc6db22b09a1 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:10.077261 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-44132df8-9443-4911-93d9-cc6db22b09a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:10.086130 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 4/7] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 20 13:22:10.355046 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2764d-5907-45d0-82a8-18ac7286a4b8 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:10.367575 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2764d-5907-45d0-82a8-18ac7286a4b8 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:10.371666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e25b03-7250-4c07-80ef-256aad2df1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:10.372251 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e25b03-7250-4c07-80ef-256aad2df1cc admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:10.372530 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e25b03-7250-4c07-80ef-256aad2df1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:10.372800 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e25b03-7250-4c07-80ef-256aad2df1cc admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:10.380190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e25b03-7250-4c07-80ef-256aad2df1cc admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:10.382041 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e25b03-7250-4c07-80ef-256aad2df1cc admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:10.390754 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 4/8] 10.222.0.23 () {58 vars in 1324 bytes} [Thu Oct 20 13:22:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 20 13:22:10.395989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44132df8-9443-4911-93d9-cc6db22b09a1 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:10.405374 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44132df8-9443-4911-93d9-cc6db22b09a1 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:10.409628 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74a077-8407-4851-bfef-c79863b268fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:10.410248 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74a077-8407-4851-bfef-c79863b268fb admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:10.410525 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74a077-8407-4851-bfef-c79863b268fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:10.410796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74a077-8407-4851-bfef-c79863b268fb admin admin] PATH_INFO: `/v3/users/b0595b7096954e6a802224bb45e3f3cf` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:10.412079 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74a077-8407-4851-bfef-c79863b268fb admin admin] RBAC: Authorizing `identity:get_user(user_id=b0595b7096954e6a802224bb45e3f3cf)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:10.414436 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74a077-8407-4851-bfef-c79863b268fb admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:10.415940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 5/9] 10.222.0.23 () {56 vars in 1198 bytes} [Thu Oct 20 13:22:10 2022] GET /identity/v3/users/b0595b7096954e6a802224bb45e3f3cf => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:12.603090 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd09cc8-c326-4b82-9f7f-d5ea560c0b36 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:12.603642 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd09cc8-c326-4b82-9f7f-d5ea560c0b36 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:12.603642 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd09cc8-c326-4b82-9f7f-d5ea560c0b36 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:12.604977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 5/10] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:12.610103 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c0332-d790-4253-8854-2f6ead779d9a None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:12.610550 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c0332-d790-4253-8854-2f6ead779d9a None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:12.610643 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c0332-d790-4253-8854-2f6ead779d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:12.628465 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-6f3c0332-d790-4253-8854-2f6ead779d9a None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:12.629678 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-6f3c0332-d790-4253-8854-2f6ead779d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:12.644454 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1765a-2880-43c4-82cb-80bab5101422 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:12.644808 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1765a-2880-43c4-82cb-80bab5101422 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:12.645137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1765a-2880-43c4-82cb-80bab5101422 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:12.646518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 6/11] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:22:12.652502 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b2fac-936c-4024-a142-9b13a05b50ef None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:12.652793 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b2fac-936c-4024-a142-9b13a05b50ef None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:12.653121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b2fac-936c-4024-a142-9b13a05b50ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:12.695366 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG passlib.handlers.bcrypt [None req-7a3b2fac-936c-4024-a142-9b13a05b50ef None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81475) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 20 13:22:12.695755 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG passlib.handlers.bcrypt [None req-7a3b2fac-936c-4024-a142-9b13a05b50ef None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81475) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 20 13:22:12.712995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 6/12] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:12.745012 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7a3b2fac-936c-4024-a142-9b13a05b50ef None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:12.746614 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7a3b2fac-936c-4024-a142-9b13a05b50ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:12.846342 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 7/13] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 13:22:12.982850 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c0332-d790-4253-8854-2f6ead779d9a None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:13.007495 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:13.087648 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c0332-d790-4253-8854-2f6ead779d9a None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:13.116718 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b2fac-936c-4024-a142-9b13a05b50ef None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:13.122176 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce814f24-dad9-4403-854d-228780fbb7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:13.122831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce814f24-dad9-4403-854d-228780fbb7de None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:13.123120 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce814f24-dad9-4403-854d-228780fbb7de None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:13.123401 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce814f24-dad9-4403-854d-228780fbb7de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:13.135190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce814f24-dad9-4403-854d-228780fbb7de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:13.144413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-ce814f24-dad9-4403-854d-228780fbb7de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:22:13.144413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:13.144413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:13.144413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:13.144413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:13.144413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:13.144413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:13.144413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:13.144413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:13.144413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:13.144413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:13.144413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:13.144413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:13.144413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:13.144413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:13.144413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:13.145358 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:13.145358 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:13.145358 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:13.145358 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:13.145358 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:13.145358 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:13.145358 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:13.145358 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:13.145358 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:13.145358 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:13.145358 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:13.145358 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:13.145358 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:13.145358 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:13.145358 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:13.145358 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:13.146143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:13.146143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:13.146143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:13.146143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:13.146143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:13.146143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:13.146143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:13.146143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:13.146143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:13.146143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:13.146143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:22:13.146143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:22:13.146143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 7/14] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 20 13:22:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:22:13.151506 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce814f24-dad9-4403-854d-228780fbb7de None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:13.217781 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b2fac-936c-4024-a142-9b13a05b50ef None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:13.220001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:13.232481 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:13.251899 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce814f24-dad9-4403-854d-228780fbb7de None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:13.257294 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b773fde-7aa8-4219-93f7-bcef1f200867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:13.258150 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b773fde-7aa8-4219-93f7-bcef1f200867 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:13.258428 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b773fde-7aa8-4219-93f7-bcef1f200867 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:13.258698 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b773fde-7aa8-4219-93f7-bcef1f200867 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:13.260061 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b773fde-7aa8-4219-93f7-bcef1f200867 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:13.261880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b773fde-7aa8-4219-93f7-bcef1f200867 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:13.273269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.sql.core [None req-8b773fde-7aa8-4219-93f7-bcef1f200867 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 20 13:22:13.273269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 20 13:22:13.273269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [parameters: {'id': '5749a36165ec4ae6851ca47601e920f8', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 20 13:22:13.273269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81475) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 20 13:22:13.285331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e3ee1-f09a-47c2-89f7-870256883725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:13.285935 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e3ee1-f09a-47c2-89f7-870256883725 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:13.286219 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e3ee1-f09a-47c2-89f7-870256883725 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:13.286495 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e3ee1-f09a-47c2-89f7-870256883725 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:13.287177 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407e3ee1-f09a-47c2-89f7-870256883725 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:13.288003 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407e3ee1-f09a-47c2-89f7-870256883725 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:13.289434 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407e3ee1-f09a-47c2-89f7-870256883725 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:13.292168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-8b773fde-7aa8-4219-93f7-bcef1f200867 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 20 13:22:13.292168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:13.292168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 13:22:13.292168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 13:22:13.292168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 13:22:13.292168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 13:22:13.292168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 13:22:13.292168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 13:22:13.292168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 13:22:13.292168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application conn.query(q) Oct 20 13:22:13.292168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 13:22:13.292168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 13:22:13.292168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 13:22:13.292168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result.read() Oct 20 13:22:13.293019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 13:22:13.293019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 13:22:13.293019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 13:22:13.293019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 13:22:13.293019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 13:22:13.293019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 13:22:13.293019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 13:22:13.293019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 13:22:13.293019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 20 13:22:13.293019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:22:13.293019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 20 13:22:13.293019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:22:13.293019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:13.293019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 20 13:22:13.293019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 20 13:22:13.293019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 20 13:22:13.293019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 20 13:22:13.293019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 13:22:13.293904 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application next(self.gen) Oct 20 13:22:13.293904 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 20 13:22:13.293904 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application yield resource Oct 20 13:22:13.293904 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 13:22:13.293904 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application next(self.gen) Oct 20 13:22:13.293904 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 20 13:22:13.293904 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.session.rollback() Oct 20 13:22:13.293904 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 20 13:22:13.293904 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.force_reraise() Oct 20 13:22:13.293904 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 20 13:22:13.293904 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise self.value Oct 20 13:22:13.293904 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 20 13:22:13.293904 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 20 13:22:13.293904 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 20 13:22:13.293904 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application session.commit() Oct 20 13:22:13.293904 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 20 13:22:13.293904 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 20 13:22:13.294748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 20 13:22:13.294748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._prepare_impl() Oct 20 13:22:13.294748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 20 13:22:13.294748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.session.flush() Oct 20 13:22:13.294748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 20 13:22:13.294748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._flush(objects) Oct 20 13:22:13.294748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 20 13:22:13.294748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 20 13:22:13.294748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 20 13:22:13.294748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application compat.raise_( Oct 20 13:22:13.294748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 13:22:13.294748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception Oct 20 13:22:13.294748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 20 13:22:13.294748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application flush_context.execute() Oct 20 13:22:13.294748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 20 13:22:13.294748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rec.execute(self) Oct 20 13:22:13.294748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 20 13:22:13.295601 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 20 13:22:13.295601 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 20 13:22:13.295601 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 20 13:22:13.295601 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 20 13:22:13.295601 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 20 13:22:13.295601 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 20 13:22:13.295601 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 20 13:22:13.295601 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 20 13:22:13.295601 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 20 13:22:13.295601 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 20 13:22:13.295601 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 20 13:22:13.295601 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 20 13:22:13.295601 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 20 13:22:13.295601 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 20 13:22:13.295601 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 20 13:22:13.296399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 13:22:13.296399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception Oct 20 13:22:13.296399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 13:22:13.296399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 13:22:13.296399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 13:22:13.296399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 13:22:13.296399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 13:22:13.296399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 13:22:13.296399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 13:22:13.296399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application conn.query(q) Oct 20 13:22:13.296399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 13:22:13.296399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 13:22:13.296399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 13:22:13.296399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result.read() Oct 20 13:22:13.296399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 13:22:13.296399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 13:22:13.296399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 13:22:13.297240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 13:22:13.297240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 13:22:13.297240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 13:22:13.297240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 13:22:13.297240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 13:22:13.297240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 20 13:22:13.297240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 20 13:22:13.297240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [parameters: {'id': '5749a36165ec4ae6851ca47601e920f8', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 20 13:22:13.297240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 20 13:22:13.297240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:22:13.297240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:22:13.297240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:22:13.297240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:13.297240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 20 13:22:13.297240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 20 13:22:13.298039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 20 13:22:13.298039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 20 13:22:13.298039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 20 13:22:13.298039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:22:13.298039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:22:13.298039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:22:13.298039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:13.298039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:13.298039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:13.298039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:13.298039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:13.298039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:13.298039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:13.298039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:13.298039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:13.298039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:13.298039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:13.298866 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 20 13:22:13.298866 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 20 13:22:13.298866 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:13.298866 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:13.298866 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 20 13:22:13.298866 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 20 13:22:13.298866 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:13.298866 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:13.298866 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 20 13:22:13.298866 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 20 13:22:13.298866 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 20 13:22:13.298866 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:22:13.298866 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 8/15] 10.222.0.23 () {60 vars in 1137 bytes} [Thu Oct 20 13:22:13 2022] POST /identity/v3/domains => generated 190 bytes in 178 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:22:13.299566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b773fde-7aa8-4219-93f7-bcef1f200867 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:13.306390 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 8/16] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:22:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:22:13.311973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e3ee1-f09a-47c2-89f7-870256883725 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:13.400853 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b773fde-7aa8-4219-93f7-bcef1f200867 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:13.411052 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e3ee1-f09a-47c2-89f7-870256883725 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:13.433949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbfb2bf-58a9-4198-970a-da52728a070a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:13.434801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfb2bf-58a9-4198-970a-da52728a070a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:13.435087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfb2bf-58a9-4198-970a-da52728a070a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:13.435361 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfb2bf-58a9-4198-970a-da52728a070a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:13.444416 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcfaf70-ddc2-4a08-bd8b-3d181aa405c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:13.445082 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcfaf70-ddc2-4a08-bd8b-3d181aa405c0 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:13.445474 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcfaf70-ddc2-4a08-bd8b-3d181aa405c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:13.445755 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcfaf70-ddc2-4a08-bd8b-3d181aa405c0 None admin] PATH_INFO: `/v3/users/b0595b7096954e6a802224bb45e3f3cf` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:13.446922 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcfaf70-ddc2-4a08-bd8b-3d181aa405c0 None admin] RBAC: Authorizing `identity:get_user(user_id=b0595b7096954e6a802224bb45e3f3cf)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:13.448388 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcfaf70-ddc2-4a08-bd8b-3d181aa405c0 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:13.450164 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 9/17] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:13 2022] GET /identity/v3/users/b0595b7096954e6a802224bb45e3f3cf => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:13.455301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcfaf70-ddc2-4a08-bd8b-3d181aa405c0 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:13.455431 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbfb2bf-58a9-4198-970a-da52728a070a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:13.456775 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbfb2bf-58a9-4198-970a-da52728a070a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:13.470000 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:13.474376 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-7cbfb2bf-58a9-4198-970a-da52728a070a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:22:13.474376 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:13.474376 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:13.474376 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:13.474376 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:13.474376 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:13.474376 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:13.474376 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:13.474376 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:13.474376 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:13.474376 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:13.474376 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:13.474376 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:22:13.474376 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:22:13.474376 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:22:13.474376 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:22:13.475393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:13.475393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:13.475393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:13.475393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:13.475393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:13.475393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:13.475393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:13.475393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:13.475393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:13.475393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:13.475393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:13.475393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:13.475393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:13.475393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:13.475393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:13.475393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:13.475393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:22:13.476380 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:22:13.476380 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:22:13.476380 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:22:13.476380 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 9/18] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:22:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:22:13.481330 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbfb2bf-58a9-4198-970a-da52728a070a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:13.504318 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:13.559613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcfaf70-ddc2-4a08-bd8b-3d181aa405c0 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:13.586854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbfb2bf-58a9-4198-970a-da52728a070a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:13.595561 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73126d9f-afeb-4346-b3d3-0820b13a4275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:13.596202 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73126d9f-afeb-4346-b3d3-0820b13a4275 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:13.596488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73126d9f-afeb-4346-b3d3-0820b13a4275 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:13.596765 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73126d9f-afeb-4346-b3d3-0820b13a4275 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:13.597921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73126d9f-afeb-4346-b3d3-0820b13a4275 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:13.599386 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73126d9f-afeb-4346-b3d3-0820b13a4275 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:13.601067 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 10/19] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:22:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:22:13.606720 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73126d9f-afeb-4346-b3d3-0820b13a4275 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:13.623595 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d1068-3e76-4444-8a5b-255c3b210576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:13.624206 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d1068-3e76-4444-8a5b-255c3b210576 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:13.624491 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d1068-3e76-4444-8a5b-255c3b210576 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:13.624769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d1068-3e76-4444-8a5b-255c3b210576 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:13.625453 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d1068-3e76-4444-8a5b-255c3b210576 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:13.625764 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d1068-3e76-4444-8a5b-255c3b210576 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:13.627144 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d1068-3e76-4444-8a5b-255c3b210576 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:13.644480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 10/20] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:13.696079 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:13.717842 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73126d9f-afeb-4346-b3d3-0820b13a4275 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:13.754013 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e1974-6d35-4fab-8841-d27cbc0f1d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:13.754690 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e1974-6d35-4fab-8841-d27cbc0f1d49 None admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:13.754975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e1974-6d35-4fab-8841-d27cbc0f1d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:13.755260 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e1974-6d35-4fab-8841-d27cbc0f1d49 None admin] PATH_INFO: `/v3/domains/default/users/b0595b7096954e6a802224bb45e3f3cf/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:13.757355 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e1974-6d35-4fab-8841-d27cbc0f1d49 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b0595b7096954e6a802224bb45e3f3cf, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:13.758844 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e1974-6d35-4fab-8841-d27cbc0f1d49 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:13.876150 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 11/21] 10.222.0.23 () {58 vars in 1361 bytes} [Thu Oct 20 13:22:13 2022] PUT /identity/v3/domains/default/users/b0595b7096954e6a802224bb45e3f3cf/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:22:16.151747 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70a5a1-3087-4737-ab2d-dcc917106770 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:16.152742 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70a5a1-3087-4737-ab2d-dcc917106770 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:16.153256 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70a5a1-3087-4737-ab2d-dcc917106770 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:16.155064 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 11/22] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:16.173713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50933675-48f4-4cef-bec1-525cd9d21ff1 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:16.174623 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50933675-48f4-4cef-bec1-525cd9d21ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:16.175285 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50933675-48f4-4cef-bec1-525cd9d21ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:16.205969 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-50933675-48f4-4cef-bec1-525cd9d21ff1 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:16.207851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-50933675-48f4-4cef-bec1-525cd9d21ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:16.289321 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 12/23] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:16.394709 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ee7d3-6431-4f18-aeb4-29d17d9f4fb7 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:16.395322 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ee7d3-6431-4f18-aeb4-29d17d9f4fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:16.395788 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ee7d3-6431-4f18-aeb4-29d17d9f4fb7 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:16.397886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 12/24] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:16.403451 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf9301-352e-4f66-badb-0ef1e8a09c3f None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:16.403987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf9301-352e-4f66-badb-0ef1e8a09c3f None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:16.404471 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf9301-352e-4f66-badb-0ef1e8a09c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:16.434141 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-9baf9301-352e-4f66-badb-0ef1e8a09c3f None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:16.435749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-9baf9301-352e-4f66-badb-0ef1e8a09c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:16.533597 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:16.538995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 13/25] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:16.584135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4e230-6fb8-49ac-8361-5123329d8f84 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:16.584908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4e230-6fb8-49ac-8361-5123329d8f84 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:16.585570 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4e230-6fb8-49ac-8361-5123329d8f84 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:16.587544 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 13/26] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:16.593506 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94178e-9ef8-4ad1-9a74-890c331abb9f None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:16.594077 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94178e-9ef8-4ad1-9a74-890c331abb9f None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:16.594555 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94178e-9ef8-4ad1-9a74-890c331abb9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:16.613808 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-4f94178e-9ef8-4ad1-9a74-890c331abb9f None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:16.615302 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-4f94178e-9ef8-4ad1-9a74-890c331abb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:16.690095 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 14/27] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:16.877148 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4e230-6fb8-49ac-8361-5123329d8f84 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:16.924346 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94178e-9ef8-4ad1-9a74-890c331abb9f None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:17.020858 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:17.054761 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4e230-6fb8-49ac-8361-5123329d8f84 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:17.089527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9fb39-f15c-4d29-a757-f8015228b60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:17.090397 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9fb39-f15c-4d29-a757-f8015228b60b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:17.090824 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9fb39-f15c-4d29-a757-f8015228b60b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:17.091236 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9fb39-f15c-4d29-a757-f8015228b60b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:17.103113 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9fb39-f15c-4d29-a757-f8015228b60b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:17.104971 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9fb39-f15c-4d29-a757-f8015228b60b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:17.108514 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94178e-9ef8-4ad1-9a74-890c331abb9f None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:17.121016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-38d9fb39-f15c-4d29-a757-f8015228b60b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:22:17.121016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:17.121016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:17.121016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:17.121016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:17.121016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:17.121016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:17.121016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:17.121016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:17.121016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:17.121016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:17.121016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:17.121016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:22:17.121016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:22:17.121016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:22:17.121016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:22:17.122079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:17.122079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:17.122079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:17.122079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:17.122079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:17.122079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:17.122079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:17.122079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:17.122079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:17.122079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:17.122079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:17.122079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:17.122079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:17.122079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:17.122079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:17.122079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:17.122079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:22:17.123069 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:22:17.123069 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:22:17.123069 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:22:17.125054 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 14/28] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:22:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:22:17.126592 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade9da5-bda2-4844-9e6b-81016062eb78 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:17.127016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade9da5-bda2-4844-9e6b-81016062eb78 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:17.127423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade9da5-bda2-4844-9e6b-81016062eb78 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:17.129122 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 15/29] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:17.130087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade9da5-bda2-4844-9e6b-81016062eb78 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:17.139331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:17.145659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aee7b2-d9c8-48bf-a6e7-0c136090a115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:17.146384 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aee7b2-d9c8-48bf-a6e7-0c136090a115 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:17.147589 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aee7b2-d9c8-48bf-a6e7-0c136090a115 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:17.147874 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aee7b2-d9c8-48bf-a6e7-0c136090a115 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:17.159032 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aee7b2-d9c8-48bf-a6e7-0c136090a115 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:17.165028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-d2aee7b2-d9c8-48bf-a6e7-0c136090a115 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:22:17.165028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:17.165028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:17.165028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:17.165028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:17.165028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:17.165028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:17.165028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:17.165028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:17.165028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:17.165028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:17.165028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:17.165028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:17.165028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:17.165028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:17.165028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:17.166148 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:17.166148 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:17.166148 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:17.166148 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:17.166148 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:17.166148 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:17.166148 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:17.166148 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:17.166148 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:17.166148 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:17.166148 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:17.166148 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:17.166148 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:17.166148 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:17.166148 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:17.166148 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:17.167065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:17.167065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:17.167065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:17.167065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:17.167065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:17.167065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:17.167065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:17.167065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:17.167065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:17.167065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:17.167065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:22:17.167065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:22:17.168622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 15/30] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 20 13:22:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:22:17.170149 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aee7b2-d9c8-48bf-a6e7-0c136090a115 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:17.254475 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade9da5-bda2-4844-9e6b-81016062eb78 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:17.292540 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aee7b2-d9c8-48bf-a6e7-0c136090a115 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:17.294072 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043ef30-7b26-4cf9-bb8c-8420d56b94b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:17.294909 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043ef30-7b26-4cf9-bb8c-8420d56b94b5 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:17.295254 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043ef30-7b26-4cf9-bb8c-8420d56b94b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:17.295598 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043ef30-7b26-4cf9-bb8c-8420d56b94b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:17.296337 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043ef30-7b26-4cf9-bb8c-8420d56b94b5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:17.297825 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043ef30-7b26-4cf9-bb8c-8420d56b94b5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:17.305112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:17.332746 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0007f43-f821-4727-9d9b-2f37a99d6f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:17.333539 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0007f43-f821-4727-9d9b-2f37a99d6f3c None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:17.333955 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0007f43-f821-4727-9d9b-2f37a99d6f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:17.334353 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0007f43-f821-4727-9d9b-2f37a99d6f3c None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:17.335124 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0007f43-f821-4727-9d9b-2f37a99d6f3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:17.335523 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0007f43-f821-4727-9d9b-2f37a99d6f3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:17.337291 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0007f43-f821-4727-9d9b-2f37a99d6f3c None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:17.359449 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 16/31] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:17.362070 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a522a3a-a2d6-41af-8e80-7c91297912b7 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:17.365991 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a522a3a-a2d6-41af-8e80-7c91297912b7 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:17.366396 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a522a3a-a2d6-41af-8e80-7c91297912b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:17.387103 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-8a522a3a-a2d6-41af-8e80-7c91297912b7 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:17.388785 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-8a522a3a-a2d6-41af-8e80-7c91297912b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:17.427484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 16/32] 10.222.0.23 () {60 vars in 1131 bytes} [Thu Oct 20 13:22:17 2022] POST /identity/v3/roles => generated 225 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:22:17.428794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043ef30-7b26-4cf9-bb8c-8420d56b94b5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:17.461260 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:17.505914 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 17/33] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:17.507522 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a522a3a-a2d6-41af-8e80-7c91297912b7 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:17.614388 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043ef30-7b26-4cf9-bb8c-8420d56b94b5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:17.670404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b383f-74e2-416c-876c-7170214f83f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:17.671036 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b383f-74e2-416c-876c-7170214f83f3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:17.671318 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b383f-74e2-416c-876c-7170214f83f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:17.671593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b383f-74e2-416c-876c-7170214f83f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:17.672272 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b383f-74e2-416c-876c-7170214f83f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:17.672575 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b383f-74e2-416c-876c-7170214f83f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:17.674132 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b383f-74e2-416c-876c-7170214f83f3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:17.684847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a522a3a-a2d6-41af-8e80-7c91297912b7 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:17.700403 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 17/34] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:22:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:17.706931 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b383f-74e2-416c-876c-7170214f83f3 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:17.731290 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef1255-9e12-44f2-b172-8cefdfd08dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:17.731913 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef1255-9e12-44f2-b172-8cefdfd08dca None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:17.732634 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef1255-9e12-44f2-b172-8cefdfd08dca None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:17.732934 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef1255-9e12-44f2-b172-8cefdfd08dca None admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:17.734421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef1255-9e12-44f2-b172-8cefdfd08dca None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:17.736784 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef1255-9e12-44f2-b172-8cefdfd08dca None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:17.793010 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 18/35] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 20 13:22:17 2022] POST /identity/v3/projects => generated 301 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:22:17.813321 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:17.861297 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b383f-74e2-416c-876c-7170214f83f3 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:17.881013 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef1255-9e12-44f2-b172-8cefdfd08dca None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:17.909212 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c1ec5-bc33-485d-ba8d-efc641828129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:17.910251 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c1ec5-bc33-485d-ba8d-efc641828129 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:17.910733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c1ec5-bc33-485d-ba8d-efc641828129 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:17.911225 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c1ec5-bc33-485d-ba8d-efc641828129 None admin] PATH_INFO: `/v3/users/b0595b7096954e6a802224bb45e3f3cf` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:17.916218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c1ec5-bc33-485d-ba8d-efc641828129 None admin] RBAC: Authorizing `identity:get_user(user_id=b0595b7096954e6a802224bb45e3f3cf)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:17.918179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c1ec5-bc33-485d-ba8d-efc641828129 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:17.920847 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 18/36] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:17 2022] GET /identity/v3/users/b0595b7096954e6a802224bb45e3f3cf => generated 247 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:22:17.923913 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:17.927513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c1ec5-bc33-485d-ba8d-efc641828129 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:18.051929 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef1255-9e12-44f2-b172-8cefdfd08dca None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:18.095359 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c1ec5-bc33-485d-ba8d-efc641828129 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:18.126527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:18.142402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1bc08e-32bd-492c-92a2-86d40dd29280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:18.143066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bc08e-32bd-492c-92a2-86d40dd29280 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:18.143354 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bc08e-32bd-492c-92a2-86d40dd29280 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:18.143631 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bc08e-32bd-492c-92a2-86d40dd29280 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:18.145158 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1bc08e-32bd-492c-92a2-86d40dd29280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:18.146719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1bc08e-32bd-492c-92a2-86d40dd29280 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:18.149424 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 19/37] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:22:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:18.153016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956b765-90ce-4681-8db8-a54f5c494946 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:18.153016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956b765-90ce-4681-8db8-a54f5c494946 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:18.153016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956b765-90ce-4681-8db8-a54f5c494946 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:18.155019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 20/38] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:18.156522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956b765-90ce-4681-8db8-a54f5c494946 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:18.200565 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc645794-c3e2-47d5-b7f5-4dfb7a1e2a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:18.201219 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc645794-c3e2-47d5-b7f5-4dfb7a1e2a4b None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:18.201502 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc645794-c3e2-47d5-b7f5-4dfb7a1e2a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:18.201781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc645794-c3e2-47d5-b7f5-4dfb7a1e2a4b None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:18.202473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc645794-c3e2-47d5-b7f5-4dfb7a1e2a4b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:18.203873 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc645794-c3e2-47d5-b7f5-4dfb7a1e2a4b None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:18.221291 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 19/39] 10.222.0.23 () {60 vars in 1131 bytes} [Thu Oct 20 13:22:17 2022] POST /identity/v3/roles => generated 219 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:22:18.222665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df51d1-9365-4391-a2cd-355bace9f884 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:18.222960 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df51d1-9365-4391-a2cd-355bace9f884 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:18.223291 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df51d1-9365-4391-a2cd-355bace9f884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:18.244682 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-a4df51d1-9365-4391-a2cd-355bace9f884 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:18.246334 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-a4df51d1-9365-4391-a2cd-355bace9f884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:18.286065 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956b765-90ce-4681-8db8-a54f5c494946 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:18.327028 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135d993-4861-4849-a8da-c23acaa88201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:18.327672 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135d993-4861-4849-a8da-c23acaa88201 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:18.327946 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135d993-4861-4849-a8da-c23acaa88201 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:18.328228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135d993-4861-4849-a8da-c23acaa88201 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:18.329015 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c135d993-4861-4849-a8da-c23acaa88201 None admin] RBAC: Adding query filter params (role.id=2bcf35e1bc5f4634b0aa6411fad2f43f, scope.domain.id=default, user.id=b0595b7096954e6a802224bb45e3f3cf) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:18.329333 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c135d993-4861-4849-a8da-c23acaa88201 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:18.330857 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c135d993-4861-4849-a8da-c23acaa88201 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:18.334799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 20/40] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:18.339318 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a608d-c8a1-4c70-b7a6-2850b1687d2d None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:18.340185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a608d-c8a1-4c70-b7a6-2850b1687d2d None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:18.340677 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a608d-c8a1-4c70-b7a6-2850b1687d2d None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:18.342170 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 21/41] 10.222.0.23 () {56 vars in 1322 bytes} [Thu Oct 20 13:22:18 2022] GET /identity/v3/role_assignments?user.id=b0595b7096954e6a802224bb45e3f3cf&scope.domain.id=default&role.id=2bcf35e1bc5f4634b0aa6411fad2f43f => generated 530 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:18.342821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 21/42] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:18.356905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20a5cf-8c9c-45e5-b59a-503c9117a71b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:18.356905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20a5cf-8c9c-45e5-b59a-503c9117a71b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:18.356905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20a5cf-8c9c-45e5-b59a-503c9117a71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:18.376733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8b20a5cf-8c9c-45e5-b59a-503c9117a71b None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:18.378517 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8b20a5cf-8c9c-45e5-b59a-503c9117a71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:18.442350 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:18.463639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 22/43] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:18.624637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a608d-c8a1-4c70-b7a6-2850b1687d2d None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:18.650070 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:18.727896 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a608d-c8a1-4c70-b7a6-2850b1687d2d None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:18.734240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b20a5cf-8c9c-45e5-b59a-503c9117a71b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:18.765043 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4d588-9fa2-430e-a204-55058a0f0d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:18.765711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4d588-9fa2-430e-a204-55058a0f0d8a None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:18.765997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4d588-9fa2-430e-a204-55058a0f0d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:18.766291 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde4d588-9fa2-430e-a204-55058a0f0d8a None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:18.766975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4d588-9fa2-430e-a204-55058a0f0d8a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:18.768363 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde4d588-9fa2-430e-a204-55058a0f0d8a None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:18.783824 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 22/44] 10.222.0.23 () {60 vars in 1131 bytes} [Thu Oct 20 13:22:18 2022] POST /identity/v3/roles => generated 223 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:22:18.841905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b20a5cf-8c9c-45e5-b59a-503c9117a71b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:18.877600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492bdb1-c206-47b2-aef0-f96ed766aca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:18.878256 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492bdb1-c206-47b2-aef0-f96ed766aca3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:18.878537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492bdb1-c206-47b2-aef0-f96ed766aca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:18.878813 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492bdb1-c206-47b2-aef0-f96ed766aca3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:18.879940 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492bdb1-c206-47b2-aef0-f96ed766aca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:18.881964 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492bdb1-c206-47b2-aef0-f96ed766aca3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:18.883560 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 23/45] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:22:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:22:18.889603 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4d588-9fa2-430e-a204-55058a0f0d8a None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:18.977015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:18.995901 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde4d588-9fa2-430e-a204-55058a0f0d8a None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:19.030000 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96020fe-c497-4dab-a485-b2fdadf0de59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:19.030610 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96020fe-c497-4dab-a485-b2fdadf0de59 None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:19.030890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96020fe-c497-4dab-a485-b2fdadf0de59 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:19.031160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96020fe-c497-4dab-a485-b2fdadf0de59 None admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:19.031810 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96020fe-c497-4dab-a485-b2fdadf0de59 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:19.033288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96020fe-c497-4dab-a485-b2fdadf0de59 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:19.058798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 23/46] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 20 13:22:18 2022] POST /identity/v3/projects => generated 312 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:22:21.239703 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea2ee9-d5e1-4a37-a095-8eb513b3c924 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:21.240216 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea2ee9-d5e1-4a37-a095-8eb513b3c924 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:21.240299 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea2ee9-d5e1-4a37-a095-8eb513b3c924 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:21.241619 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 24/47] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:21.247310 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b901e23-6b39-4166-8881-400dcc8d9909 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:21.247669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b901e23-6b39-4166-8881-400dcc8d9909 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:21.247872 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b901e23-6b39-4166-8881-400dcc8d9909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:21.266816 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-3b901e23-6b39-4166-8881-400dcc8d9909 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:21.267994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-3b901e23-6b39-4166-8881-400dcc8d9909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:21.333126 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 24/48] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:21.601679 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea2ee9-d5e1-4a37-a095-8eb513b3c924 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:21.655067 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:21.703183 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea2ee9-d5e1-4a37-a095-8eb513b3c924 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:21.736234 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bf35d-2577-477e-a15f-f5d1ac277dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:21.736854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bf35d-2577-477e-a15f-f5d1ac277dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:21.737166 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bf35d-2577-477e-a15f-f5d1ac277dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:21.737441 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bf35d-2577-477e-a15f-f5d1ac277dc1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:21.738543 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bf35d-2577-477e-a15f-f5d1ac277dc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:21.739974 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bf35d-2577-477e-a15f-f5d1ac277dc1 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:21.742029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 25/49] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:22:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:21.747625 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b901e23-6b39-4166-8881-400dcc8d9909 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:21.807564 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:21.845202 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b901e23-6b39-4166-8881-400dcc8d9909 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:21.877151 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10e5e0-07fa-4aa5-92b9-e11d31f226f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:21.877750 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10e5e0-07fa-4aa5-92b9-e11d31f226f6 None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:21.878028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10e5e0-07fa-4aa5-92b9-e11d31f226f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:21.878297 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10e5e0-07fa-4aa5-92b9-e11d31f226f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:21.878948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b10e5e0-07fa-4aa5-92b9-e11d31f226f6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:21.880385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b10e5e0-07fa-4aa5-92b9-e11d31f226f6 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:21.905898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 25/50] 10.222.0.23 () {60 vars in 1140 bytes} [Thu Oct 20 13:22:21 2022] POST /identity/v3/projects => generated 298 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:22:24.033405 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a2996-9c45-42fa-bff4-3d48e5574ecd None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:24.033918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a2996-9c45-42fa-bff4-3d48e5574ecd None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:24.034027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a2996-9c45-42fa-bff4-3d48e5574ecd None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:24.035295 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 26/51] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:24.040707 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97696c-60cb-4c6d-8a29-7c3fa1fd8935 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:24.041136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97696c-60cb-4c6d-8a29-7c3fa1fd8935 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:24.041309 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97696c-60cb-4c6d-8a29-7c3fa1fd8935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:24.059835 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-cb97696c-60cb-4c6d-8a29-7c3fa1fd8935 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:24.061050 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-cb97696c-60cb-4c6d-8a29-7c3fa1fd8935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:24.124001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 26/52] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:24.392898 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a2996-9c45-42fa-bff4-3d48e5574ecd None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:24.491200 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a2996-9c45-42fa-bff4-3d48e5574ecd None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:24.510915 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:24.525910 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af8d37-91c1-4790-8d7b-3f11640785e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:24.526536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af8d37-91c1-4790-8d7b-3f11640785e3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:24.526820 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af8d37-91c1-4790-8d7b-3f11640785e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:24.527097 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af8d37-91c1-4790-8d7b-3f11640785e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:24.528207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af8d37-91c1-4790-8d7b-3f11640785e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:24.529672 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af8d37-91c1-4790-8d7b-3f11640785e3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:24.531226 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 27/53] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:22:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:24.536821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97696c-60cb-4c6d-8a29-7c3fa1fd8935 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:24.580742 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:24.632555 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97696c-60cb-4c6d-8a29-7c3fa1fd8935 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:24.665299 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da9baf-c5a4-4ff6-a2b9-cb34e9baaa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:24.665836 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da9baf-c5a4-4ff6-a2b9-cb34e9baaa96 None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:24.666117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da9baf-c5a4-4ff6-a2b9-cb34e9baaa96 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:24.666388 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da9baf-c5a4-4ff6-a2b9-cb34e9baaa96 None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:24.667039 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da9baf-c5a4-4ff6-a2b9-cb34e9baaa96 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:24.668478 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da9baf-c5a4-4ff6-a2b9-cb34e9baaa96 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:24.695045 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 27/54] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 20 13:22:24 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:22:27.899168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb6af8-f3b3-4345-b946-4b508edd5574 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:27.900167 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb6af8-f3b3-4345-b946-4b508edd5574 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:27.900643 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb6af8-f3b3-4345-b946-4b508edd5574 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:27.902501 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 28/55] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:27.907365 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6051e3d-81e6-4298-9a0d-ab9408fd46e3 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:27.907750 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6051e3d-81e6-4298-9a0d-ab9408fd46e3 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:27.907916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6051e3d-81e6-4298-9a0d-ab9408fd46e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:27.933341 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-d6051e3d-81e6-4298-9a0d-ab9408fd46e3 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:27.934824 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-d6051e3d-81e6-4298-9a0d-ab9408fd46e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:28.035682 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 28/56] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:28.146258 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64483e4d-3a55-41d8-96c4-e838f73258ae None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:28.147001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64483e4d-3a55-41d8-96c4-e838f73258ae None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:28.147473 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64483e4d-3a55-41d8-96c4-e838f73258ae None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:28.149040 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 29/57] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:28.154594 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a9660-fd51-415d-8c8f-3a3db8e4fc7e None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:28.155051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a9660-fd51-415d-8c8f-3a3db8e4fc7e None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:28.155485 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a9660-fd51-415d-8c8f-3a3db8e4fc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:28.183801 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-2e7a9660-fd51-415d-8c8f-3a3db8e4fc7e None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:28.185303 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-2e7a9660-fd51-415d-8c8f-3a3db8e4fc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:28.227501 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:28.287775 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 29/58] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:28.378127 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb26231-70eb-4e75-a8a7-8f498f3567c0 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:28.378748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb26231-70eb-4e75-a8a7-8f498f3567c0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:28.379223 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb26231-70eb-4e75-a8a7-8f498f3567c0 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:28.380763 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 30/59] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:28.386386 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad496f7f-a44c-4ef0-b223-87f36b5572ab None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:28.386983 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad496f7f-a44c-4ef0-b223-87f36b5572ab None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:28.387712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad496f7f-a44c-4ef0-b223-87f36b5572ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:28.413961 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-ad496f7f-a44c-4ef0-b223-87f36b5572ab None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:28.415474 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-ad496f7f-a44c-4ef0-b223-87f36b5572ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:28.425932 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb26231-70eb-4e75-a8a7-8f498f3567c0 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:28.505214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 30/60] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:28.507350 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0e01e-26d5-41e7-a79c-cf3eea80902d None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:28.507868 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0e01e-26d5-41e7-a79c-cf3eea80902d None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:28.508341 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0e01e-26d5-41e7-a79c-cf3eea80902d None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:28.514003 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 31/61] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:28.526539 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b812d-a867-45c6-8612-f8f8bff4ee54 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:28.527090 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b812d-a867-45c6-8612-f8f8bff4ee54 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:28.527562 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b812d-a867-45c6-8612-f8f8bff4ee54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:28.564983 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-5a0b812d-a867-45c6-8612-f8f8bff4ee54 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:28.567325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-5a0b812d-a867-45c6-8612-f8f8bff4ee54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:28.569944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb26231-70eb-4e75-a8a7-8f498f3567c0 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:28.627763 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f840570d-b0c4-4edf-bb0d-21b0f8699797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:28.628459 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840570d-b0c4-4edf-bb0d-21b0f8699797 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:28.628748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840570d-b0c4-4edf-bb0d-21b0f8699797 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:28.629050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840570d-b0c4-4edf-bb0d-21b0f8699797 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:28.654908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f840570d-b0c4-4edf-bb0d-21b0f8699797 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:28.662737 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-f840570d-b0c4-4edf-bb0d-21b0f8699797 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:22:28.662737 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:28.662737 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:28.662737 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:28.662737 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:28.662737 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:28.662737 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:28.662737 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:28.662737 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:28.662737 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:28.662737 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:28.662737 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:28.662737 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:28.662737 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:28.662737 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:28.662737 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:28.663692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:28.663692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:28.663692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:28.663692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:28.663692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:28.663692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:28.663692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:28.663692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:28.663692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:28.663692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:28.663692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:28.663692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:28.663692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:28.663692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:28.663692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:28.663692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:28.664532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:28.664532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:28.664532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:28.664532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:28.664532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:28.664532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:28.664532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:28.664532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:28.664532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:28.664532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:28.664532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:22:28.664532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:22:28.664532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 31/62] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:22:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:22:28.665204 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f840570d-b0c4-4edf-bb0d-21b0f8699797 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:28.695620 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 32/63] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:28.696774 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b812d-a867-45c6-8612-f8f8bff4ee54 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:28.727675 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:28.731455 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:28.784914 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f840570d-b0c4-4edf-bb0d-21b0f8699797 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:28.821636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b812d-a867-45c6-8612-f8f8bff4ee54 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:28.823059 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec801e-00e7-47e2-967f-5b52184d9c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:28.823969 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec801e-00e7-47e2-967f-5b52184d9c8b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:28.825040 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec801e-00e7-47e2-967f-5b52184d9c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:28.825325 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec801e-00e7-47e2-967f-5b52184d9c8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:28.837430 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec801e-00e7-47e2-967f-5b52184d9c8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:28.843664 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-3aec801e-00e7-47e2-967f-5b52184d9c8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:22:28.843664 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:28.843664 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:28.843664 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:28.843664 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:28.843664 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:28.843664 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:28.843664 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:28.843664 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:28.843664 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:28.843664 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:28.843664 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:28.843664 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:28.843664 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:28.843664 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:28.843664 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:28.844519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:28.844519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:28.844519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:28.844519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:28.844519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:28.844519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:28.844519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:28.844519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:28.844519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:28.844519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:28.844519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:28.844519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:28.844519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:28.844519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:28.844519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:28.844519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:28.845868 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:28.845868 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:28.845868 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:28.845868 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:28.845868 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:28.845868 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:28.845868 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:28.845868 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:28.845868 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:28.845868 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:28.845868 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:22:28.845868 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:22:28.845868 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 32/64] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 20 13:22:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:22:28.845868 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278442a8-074a-4060-9168-450038c2bc65 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:28.846614 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278442a8-074a-4060-9168-450038c2bc65 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:28.846614 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278442a8-074a-4060-9168-450038c2bc65 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:28.847701 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 33/65] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:28.852846 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278442a8-074a-4060-9168-450038c2bc65 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:28.866552 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196204a-d42d-4350-b66b-a5af7355ce96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:28.867192 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196204a-d42d-4350-b66b-a5af7355ce96 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:28.867479 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196204a-d42d-4350-b66b-a5af7355ce96 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:28.867756 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196204a-d42d-4350-b66b-a5af7355ce96 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:28.868455 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e196204a-d42d-4350-b66b-a5af7355ce96 None admin] RBAC: Adding query filter params (name=member) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:28.868768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e196204a-d42d-4350-b66b-a5af7355ce96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:28.870254 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e196204a-d42d-4350-b66b-a5af7355ce96 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:28.885201 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 33/66] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 20 13:22:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:28.885201 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196204a-d42d-4350-b66b-a5af7355ce96 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:28.980452 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:28.997967 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196204a-d42d-4350-b66b-a5af7355ce96 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:29.030554 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68d29a-9ab1-48b8-98ea-aa923118ce23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:29.031154 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68d29a-9ab1-48b8-98ea-aa923118ce23 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:29.031438 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68d29a-9ab1-48b8-98ea-aa923118ce23 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:29.031711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68d29a-9ab1-48b8-98ea-aa923118ce23 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:29.032370 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68d29a-9ab1-48b8-98ea-aa923118ce23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:29.032671 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68d29a-9ab1-48b8-98ea-aa923118ce23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:29.034091 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68d29a-9ab1-48b8-98ea-aa923118ce23 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:29.041875 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 34/67] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:22:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:29.043001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a3f5b-4d74-42d0-b41c-c23d61c3598d None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:29.043284 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a3f5b-4d74-42d0-b41c-c23d61c3598d None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:29.043561 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a3f5b-4d74-42d0-b41c-c23d61c3598d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:29.061258 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-b43a3f5b-4d74-42d0-b41c-c23d61c3598d None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:29.062427 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b43a3f5b-4d74-42d0-b41c-c23d61c3598d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:29.069784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278442a8-074a-4060-9168-450038c2bc65 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:29.078948 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:29.103444 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc631e64-0013-4f2c-a692-1959d0e782af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:29.104063 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc631e64-0013-4f2c-a692-1959d0e782af None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:29.104348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc631e64-0013-4f2c-a692-1959d0e782af None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:29.104627 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc631e64-0013-4f2c-a692-1959d0e782af None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:29.114991 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc631e64-0013-4f2c-a692-1959d0e782af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:29.120328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-cc631e64-0013-4f2c-a692-1959d0e782af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:22:29.120328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:29.120328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:29.120328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:29.120328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:29.120328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:29.120328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:29.120328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:29.120328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:29.120328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:29.120328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:29.120328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:29.120328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:29.120328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:29.120328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:29.120328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:29.121249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:29.121249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:29.121249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:29.121249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:29.121249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:29.121249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:29.121249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:29.121249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:29.121249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:29.121249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:29.121249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:29.121249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:29.121249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:29.121249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:29.121249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:29.121249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:29.122007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:29.122007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:29.122007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:29.122007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:29.122007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:29.122007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:29.122007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:29.122007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:29.122007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:29.122007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:29.122007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:22:29.122007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:22:29.122007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 34/68] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:22:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:22:29.122771 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc631e64-0013-4f2c-a692-1959d0e782af None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:29.133810 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 35/69] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:29.134659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a3f5b-4d74-42d0-b41c-c23d61c3598d None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:29.226389 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:29.231301 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc631e64-0013-4f2c-a692-1959d0e782af None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:29.252268 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a3f5b-4d74-42d0-b41c-c23d61c3598d None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:29.268299 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9427787-cb48-4e51-a937-346912dd9927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:29.269267 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9427787-cb48-4e51-a937-346912dd9927 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:29.269736 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9427787-cb48-4e51-a937-346912dd9927 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:29.270206 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9427787-cb48-4e51-a937-346912dd9927 None admin] PATH_INFO: `/v3/users/b7bc816de9e04a01adb1bc809287ea78` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:29.286848 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9427787-cb48-4e51-a937-346912dd9927 None admin] RBAC: Authorizing `identity:get_user(user_id=b7bc816de9e04a01adb1bc809287ea78)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:29.286987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f47a65-1505-46a6-8fbd-5fb7b4b7b1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:29.288647 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f47a65-1505-46a6-8fbd-5fb7b4b7b1db None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:29.288959 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f47a65-1505-46a6-8fbd-5fb7b4b7b1db None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:29.289240 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f47a65-1505-46a6-8fbd-5fb7b4b7b1db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:29.289644 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9427787-cb48-4e51-a937-346912dd9927 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:29.291540 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 35/70] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:29 2022] GET /identity/v3/users/b7bc816de9e04a01adb1bc809287ea78 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:29.292561 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9427787-cb48-4e51-a937-346912dd9927 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:29.305240 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f47a65-1505-46a6-8fbd-5fb7b4b7b1db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:29.310274 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-84f47a65-1505-46a6-8fbd-5fb7b4b7b1db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 13:22:29.310274 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:29.310274 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:29.310274 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:29.310274 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:29.310274 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:29.310274 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:29.310274 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:29.310274 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:29.310274 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:29.310274 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:29.310274 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:29.310274 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:29.310274 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:29.310274 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:29.310274 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:29.311102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:29.311102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:29.311102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:29.311102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:29.311102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:29.311102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:29.311102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:29.311102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:29.311102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:29.311102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:29.311102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:29.311102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:29.311102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:29.311102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:29.311102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:29.311102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:29.311833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:29.311833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:29.311833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:29.311833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:29.311833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:29.311833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:29.311833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:29.311833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:29.311833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:29.311833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:29.311833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 13:22:29.311833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:22:29.311833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 36/71] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 20 13:22:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:22:29.312461 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f47a65-1505-46a6-8fbd-5fb7b4b7b1db None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:29.322915 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:29.421648 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9427787-cb48-4e51-a937-346912dd9927 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:29.446412 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f47a65-1505-46a6-8fbd-5fb7b4b7b1db None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:29.460708 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37cd34-4ae1-4a7c-9540-4967f4cea806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:29.461361 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37cd34-4ae1-4a7c-9540-4967f4cea806 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:29.461644 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37cd34-4ae1-4a7c-9540-4967f4cea806 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:29.461918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37cd34-4ae1-4a7c-9540-4967f4cea806 None admin] PATH_INFO: `/v3/users/b0595b7096954e6a802224bb45e3f3cf` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:29.463249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a37cd34-4ae1-4a7c-9540-4967f4cea806 None admin] RBAC: Authorizing `identity:get_user(user_id=b0595b7096954e6a802224bb45e3f3cf)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:29.465040 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a37cd34-4ae1-4a7c-9540-4967f4cea806 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:29.466639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 36/72] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:29 2022] GET /identity/v3/users/b0595b7096954e6a802224bb45e3f3cf => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:29.467460 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37cd34-4ae1-4a7c-9540-4967f4cea806 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:29.479988 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996e9b4-1378-42ed-8778-ce1ea5623474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:29.480599 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996e9b4-1378-42ed-8778-ce1ea5623474 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:29.480901 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996e9b4-1378-42ed-8778-ce1ea5623474 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:29.481175 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996e9b4-1378-42ed-8778-ce1ea5623474 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:29.481884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996e9b4-1378-42ed-8778-ce1ea5623474 None admin] RBAC: Adding query filter params (name=member) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:29.482138 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996e9b4-1378-42ed-8778-ce1ea5623474 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:29.483522 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996e9b4-1378-42ed-8778-ce1ea5623474 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:29.485655 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:29.492660 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 37/73] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 20 13:22:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:29.493488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996e9b4-1378-42ed-8778-ce1ea5623474 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:29.565020 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:29.572362 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37cd34-4ae1-4a7c-9540-4967f4cea806 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:29.596470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996e9b4-1378-42ed-8778-ce1ea5623474 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:29.607949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c9e5b-9284-4353-9fb6-b531ea5821e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:29.608578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c9e5b-9284-4353-9fb6-b531ea5821e3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:29.609053 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c9e5b-9284-4353-9fb6-b531ea5821e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:29.609330 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c9e5b-9284-4353-9fb6-b531ea5821e3 None admin] PATH_INFO: `/v3/projects/2a7d45346e944c6089c645b9c6958126` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:29.610469 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c9e5b-9284-4353-9fb6-b531ea5821e3 None admin] RBAC: Authorizing `identity:get_project(project_id=2a7d45346e944c6089c645b9c6958126)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:29.611896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c9e5b-9284-4353-9fb6-b531ea5821e3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:29.613545 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 37/74] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:22:29 2022] GET /identity/v3/projects/2a7d45346e944c6089c645b9c6958126 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:29.614300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c9e5b-9284-4353-9fb6-b531ea5821e3 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:29.634050 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d008e5-d64a-41fa-8653-12e88e542cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:29.634661 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d008e5-d64a-41fa-8653-12e88e542cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:29.634946 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d008e5-d64a-41fa-8653-12e88e542cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:29.635223 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d008e5-d64a-41fa-8653-12e88e542cc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:29.635889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d008e5-d64a-41fa-8653-12e88e542cc5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:29.636201 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d008e5-d64a-41fa-8653-12e88e542cc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:29.637881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d008e5-d64a-41fa-8653-12e88e542cc5 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:29.645899 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 38/75] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:22:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:29.646832 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d008e5-d64a-41fa-8653-12e88e542cc5 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:29.703960 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:29.715202 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c9e5b-9284-4353-9fb6-b531ea5821e3 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:29.745329 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d008e5-d64a-41fa-8653-12e88e542cc5 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:29.749269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffabf33-3006-405c-a3d8-ed21001c386c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:29.749891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffabf33-3006-405c-a3d8-ed21001c386c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:29.750169 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffabf33-3006-405c-a3d8-ed21001c386c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:29.750438 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffabf33-3006-405c-a3d8-ed21001c386c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:29.751558 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffabf33-3006-405c-a3d8-ed21001c386c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:29.753014 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffabf33-3006-405c-a3d8-ed21001c386c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:29.754736 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 38/76] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:22:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:29.755317 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffabf33-3006-405c-a3d8-ed21001c386c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:29.779489 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077db79-82c5-4527-a6ed-4688fe40cf37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:29.780114 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077db79-82c5-4527-a6ed-4688fe40cf37 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:29.780398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077db79-82c5-4527-a6ed-4688fe40cf37 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:29.780795 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077db79-82c5-4527-a6ed-4688fe40cf37 None admin] PATH_INFO: `/v3/projects/2a7d45346e944c6089c645b9c6958126` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:29.782395 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077db79-82c5-4527-a6ed-4688fe40cf37 None admin] RBAC: Authorizing `identity:get_project(project_id=2a7d45346e944c6089c645b9c6958126)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:29.783842 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077db79-82c5-4527-a6ed-4688fe40cf37 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:29.785499 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 39/77] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:22:29 2022] GET /identity/v3/projects/2a7d45346e944c6089c645b9c6958126 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:29.786232 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077db79-82c5-4527-a6ed-4688fe40cf37 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:29.791418 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:29.856729 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffabf33-3006-405c-a3d8-ed21001c386c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:29.886573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077db79-82c5-4527-a6ed-4688fe40cf37 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:29.890796 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c73be7-87a3-4ef6-afb6-f87634e4adeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:29.891429 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c73be7-87a3-4ef6-afb6-f87634e4adeb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:29.891715 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c73be7-87a3-4ef6-afb6-f87634e4adeb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:29.891992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c73be7-87a3-4ef6-afb6-f87634e4adeb None admin] PATH_INFO: `/v3/users/b7bc816de9e04a01adb1bc809287ea78` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:29.893163 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c73be7-87a3-4ef6-afb6-f87634e4adeb None admin] RBAC: Authorizing `identity:get_user(user_id=b7bc816de9e04a01adb1bc809287ea78)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:29.894608 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c73be7-87a3-4ef6-afb6-f87634e4adeb None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:29.896184 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 39/78] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:29 2022] GET /identity/v3/users/b7bc816de9e04a01adb1bc809287ea78 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:29.897061 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c73be7-87a3-4ef6-afb6-f87634e4adeb None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:29.912718 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:29.920777 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb184f34-d286-4ba4-811e-815a8fd99789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:29.921474 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb184f34-d286-4ba4-811e-815a8fd99789 None admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:29.921755 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb184f34-d286-4ba4-811e-815a8fd99789 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:29.922032 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb184f34-d286-4ba4-811e-815a8fd99789 None admin] PATH_INFO: `/v3/projects/2a7d45346e944c6089c645b9c6958126/users/b7bc816de9e04a01adb1bc809287ea78/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:29.923935 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb184f34-d286-4ba4-811e-815a8fd99789 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a7d45346e944c6089c645b9c6958126, user_id=b7bc816de9e04a01adb1bc809287ea78, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:29.925423 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb184f34-d286-4ba4-811e-815a8fd99789 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:29.938734 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 40/79] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:22:29 2022] PUT /identity/v3/projects/2a7d45346e944c6089c645b9c6958126/users/b7bc816de9e04a01adb1bc809287ea78/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:22:29.939520 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb184f34-d286-4ba4-811e-815a8fd99789 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:30.016715 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:30.021430 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c73be7-87a3-4ef6-afb6-f87634e4adeb None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:30.052810 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb184f34-d286-4ba4-811e-815a8fd99789 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:30.062316 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1747046-4348-4377-81dc-dfb0e6fcbb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:30.062967 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1747046-4348-4377-81dc-dfb0e6fcbb24 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:30.063248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1747046-4348-4377-81dc-dfb0e6fcbb24 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:30.063520 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1747046-4348-4377-81dc-dfb0e6fcbb24 None admin] PATH_INFO: `/v3/users/b7bc816de9e04a01adb1bc809287ea78` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:30.064776 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1747046-4348-4377-81dc-dfb0e6fcbb24 None admin] RBAC: Authorizing `identity:get_user(user_id=b7bc816de9e04a01adb1bc809287ea78)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:30.066307 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1747046-4348-4377-81dc-dfb0e6fcbb24 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:30.073192 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 40/80] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:29 2022] GET /identity/v3/users/b7bc816de9e04a01adb1bc809287ea78 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:30.073192 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1747046-4348-4377-81dc-dfb0e6fcbb24 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:30.092366 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274200c1-8396-4c15-88ee-e5a1b625e491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:30.093027 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274200c1-8396-4c15-88ee-e5a1b625e491 None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:30.093316 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274200c1-8396-4c15-88ee-e5a1b625e491 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:30.093578 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274200c1-8396-4c15-88ee-e5a1b625e491 None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:30.094249 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274200c1-8396-4c15-88ee-e5a1b625e491 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:30.095729 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274200c1-8396-4c15-88ee-e5a1b625e491 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:30.124446 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 41/81] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 20 13:22:29 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:22:30.125331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274200c1-8396-4c15-88ee-e5a1b625e491 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:30.183788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:30.206437 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1747046-4348-4377-81dc-dfb0e6fcbb24 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:30.253582 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274200c1-8396-4c15-88ee-e5a1b625e491 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:30.258963 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e98ddc-51df-4a5f-92d6-2617e0143e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:30.259659 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e98ddc-51df-4a5f-92d6-2617e0143e79 None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:30.259944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e98ddc-51df-4a5f-92d6-2617e0143e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:30.260224 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e98ddc-51df-4a5f-92d6-2617e0143e79 None admin] PATH_INFO: `/v3/projects/2a7d45346e944c6089c645b9c6958126/users/b0595b7096954e6a802224bb45e3f3cf/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:30.262358 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e98ddc-51df-4a5f-92d6-2617e0143e79 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a7d45346e944c6089c645b9c6958126, user_id=b0595b7096954e6a802224bb45e3f3cf, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:30.264095 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e98ddc-51df-4a5f-92d6-2617e0143e79 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:30.297294 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 41/82] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:22:30 2022] PUT /identity/v3/projects/2a7d45346e944c6089c645b9c6958126/users/b0595b7096954e6a802224bb45e3f3cf/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:22:30.298626 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e98ddc-51df-4a5f-92d6-2617e0143e79 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:30.307277 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00940ad0-c11e-4157-badf-2460ccf146db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:30.307951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00940ad0-c11e-4157-badf-2460ccf146db None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:30.308245 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00940ad0-c11e-4157-badf-2460ccf146db None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:30.308528 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00940ad0-c11e-4157-badf-2460ccf146db None admin] PATH_INFO: `/v3/projects/21788bc08f594f52839376f2dd54ddba` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:30.309819 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00940ad0-c11e-4157-badf-2460ccf146db None admin] RBAC: Authorizing `identity:get_project(project_id=21788bc08f594f52839376f2dd54ddba)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:30.316891 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00940ad0-c11e-4157-badf-2460ccf146db None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:30.316891 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 42/83] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:22:30 2022] GET /identity/v3/projects/21788bc08f594f52839376f2dd54ddba => generated 312 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:30.322140 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00940ad0-c11e-4157-badf-2460ccf146db None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:30.365276 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:30.444423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e98ddc-51df-4a5f-92d6-2617e0143e79 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:30.492243 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00940ad0-c11e-4157-badf-2460ccf146db None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:30.510330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:30.511529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b4290-89e9-43ff-b021-a0bf21caa9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:30.513348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b4290-89e9-43ff-b021-a0bf21caa9b1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:30.514001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b4290-89e9-43ff-b021-a0bf21caa9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:30.514437 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b4290-89e9-43ff-b021-a0bf21caa9b1 None admin] PATH_INFO: `/v3/projects/2a7d45346e944c6089c645b9c6958126` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:30.516177 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b4290-89e9-43ff-b021-a0bf21caa9b1 None admin] RBAC: Authorizing `identity:get_project(project_id=2a7d45346e944c6089c645b9c6958126)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:30.518092 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b4290-89e9-43ff-b021-a0bf21caa9b1 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:30.520898 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 42/84] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:22:30 2022] GET /identity/v3/projects/2a7d45346e944c6089c645b9c6958126 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:30.530005 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b4290-89e9-43ff-b021-a0bf21caa9b1 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:30.545706 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3709bfc-7df6-4f9e-a764-4a53c3906b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:30.545706 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3709bfc-7df6-4f9e-a764-4a53c3906b9f None admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:30.546006 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3709bfc-7df6-4f9e-a764-4a53c3906b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:30.546124 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3709bfc-7df6-4f9e-a764-4a53c3906b9f None admin] PATH_INFO: `/v3/projects/21788bc08f594f52839376f2dd54ddba/users/b7bc816de9e04a01adb1bc809287ea78/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:30.549721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3709bfc-7df6-4f9e-a764-4a53c3906b9f None admin] RBAC: Authorizing `identity:create_grant(project_id=21788bc08f594f52839376f2dd54ddba, user_id=b7bc816de9e04a01adb1bc809287ea78, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:30.551183 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3709bfc-7df6-4f9e-a764-4a53c3906b9f None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:30.571065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 43/85] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:22:30 2022] PUT /identity/v3/projects/21788bc08f594f52839376f2dd54ddba/users/b7bc816de9e04a01adb1bc809287ea78/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:22:30.649396 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:30.659590 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b4290-89e9-43ff-b021-a0bf21caa9b1 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:30.699618 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e8be0-f25f-4f46-8b3d-77c696f130f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:30.700307 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e8be0-f25f-4f46-8b3d-77c696f130f0 None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:30.700592 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e8be0-f25f-4f46-8b3d-77c696f130f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:30.700891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e8be0-f25f-4f46-8b3d-77c696f130f0 None admin] PATH_INFO: `/v3/projects/2a7d45346e944c6089c645b9c6958126/users/b7bc816de9e04a01adb1bc809287ea78/roles/b0ccf031da9e4a7492c567b607ceb21f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:30.705615 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e8be0-f25f-4f46-8b3d-77c696f130f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a7d45346e944c6089c645b9c6958126, user_id=b7bc816de9e04a01adb1bc809287ea78, role_id=b0ccf031da9e4a7492c567b607ceb21f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:30.707126 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e8be0-f25f-4f46-8b3d-77c696f130f0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:30.727623 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 43/86] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:22:30 2022] PUT /identity/v3/projects/2a7d45346e944c6089c645b9c6958126/users/b7bc816de9e04a01adb1bc809287ea78/roles/b0ccf031da9e4a7492c567b607ceb21f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:22:32.983301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b6729-198d-4188-9d1f-b333e388ca21 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:32.984311 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b6729-198d-4188-9d1f-b333e388ca21 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:32.984786 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b6729-198d-4188-9d1f-b333e388ca21 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:32.986506 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 44/87] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:32.998371 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903e6d5-17b8-4d5a-9de6-e26e38dfaa0b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:32.999272 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903e6d5-17b8-4d5a-9de6-e26e38dfaa0b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:32.999754 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903e6d5-17b8-4d5a-9de6-e26e38dfaa0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:33.022567 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b903e6d5-17b8-4d5a-9de6-e26e38dfaa0b None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:33.023767 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b903e6d5-17b8-4d5a-9de6-e26e38dfaa0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:33.104781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecf25a-7792-4df7-93d5-90ef84d4c48a None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:33.105483 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecf25a-7792-4df7-93d5-90ef84d4c48a None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:33.105945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecf25a-7792-4df7-93d5-90ef84d4c48a None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:33.107442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 45/88] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:22:33.110010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 44/89] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:33.124564 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae28bf0-80f3-483b-b47f-f12da3ec14d4 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:33.125171 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae28bf0-80f3-483b-b47f-f12da3ec14d4 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:33.125640 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae28bf0-80f3-483b-b47f-f12da3ec14d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:33.149606 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-dae28bf0-80f3-483b-b47f-f12da3ec14d4 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:33.151123 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-dae28bf0-80f3-483b-b47f-f12da3ec14d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:33.250689 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 46/90] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:33.382258 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb500d5-93d3-4e67-b55b-40f6f29c8054 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:33.382575 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb500d5-93d3-4e67-b55b-40f6f29c8054 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:33.382823 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb500d5-93d3-4e67-b55b-40f6f29c8054 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:33.384262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 45/91] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:33.389791 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f7a55-5912-4570-afe8-96d5dd79b872 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:33.390082 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f7a55-5912-4570-afe8-96d5dd79b872 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:33.390362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f7a55-5912-4570-afe8-96d5dd79b872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:33.418141 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-3b8f7a55-5912-4570-afe8-96d5dd79b872 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:33.419407 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-3b8f7a55-5912-4570-afe8-96d5dd79b872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:33.514384 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 47/92] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:33.548673 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb500d5-93d3-4e67-b55b-40f6f29c8054 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:33.670564 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:33.692672 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb500d5-93d3-4e67-b55b-40f6f29c8054 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:33.732679 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f7a55-5912-4570-afe8-96d5dd79b872 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:33.757161 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f08dc-8549-4e93-a8f4-97924401c741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:33.757161 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f08dc-8549-4e93-a8f4-97924401c741 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:33.757161 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f08dc-8549-4e93-a8f4-97924401c741 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:33.757161 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f08dc-8549-4e93-a8f4-97924401c741 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:33.761313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f08dc-8549-4e93-a8f4-97924401c741 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:33.766896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-901f08dc-8549-4e93-a8f4-97924401c741 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:22:33.766896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:33.766896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:33.766896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:33.766896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:33.766896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:33.766896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:33.766896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:33.766896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:33.766896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:33.766896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:33.766896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:33.766896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:33.766896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:33.766896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:33.766896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:33.767728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:33.767728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:33.767728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:33.767728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:33.767728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:33.767728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:33.767728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:33.767728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:33.767728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:33.767728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:33.767728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:33.767728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:33.767728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:33.767728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:33.767728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:33.767728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:33.768472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:33.768472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:33.768472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:33.768472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:33.768472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:33.768472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:33.768472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:33.768472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:33.768472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:33.768472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:33.768472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:22:33.768472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:22:33.768472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 46/93] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 20 13:22:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:22:33.780903 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f08dc-8549-4e93-a8f4-97924401c741 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:33.802282 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:33.910113 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f08dc-8549-4e93-a8f4-97924401c741 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:33.920561 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f7a55-5912-4570-afe8-96d5dd79b872 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:33.948805 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807984fc-2f43-4701-a18c-485c0ea214b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:33.949521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807984fc-2f43-4701-a18c-485c0ea214b5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:33.949809 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807984fc-2f43-4701-a18c-485c0ea214b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:33.950086 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807984fc-2f43-4701-a18c-485c0ea214b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:33.950756 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807984fc-2f43-4701-a18c-485c0ea214b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:33.951062 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807984fc-2f43-4701-a18c-485c0ea214b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:33.952622 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807984fc-2f43-4701-a18c-485c0ea214b5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:33.962671 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 47/94] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:22:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:22:33.963981 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae456ea-d1c1-4887-87e1-7be66fb175ce None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:33.964271 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae456ea-d1c1-4887-87e1-7be66fb175ce None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:33.964548 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae456ea-d1c1-4887-87e1-7be66fb175ce None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:33.966185 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 48/95] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:33.970881 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae456ea-d1c1-4887-87e1-7be66fb175ce None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:33.971025 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29a3bb-f652-4d93-8f28-f2788f6bff8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:33.971025 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29a3bb-f652-4d93-8f28-f2788f6bff8a None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:33.971025 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29a3bb-f652-4d93-8f28-f2788f6bff8a None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:33.971025 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29a3bb-f652-4d93-8f28-f2788f6bff8a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:33.982089 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:33.984335 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e29a3bb-f652-4d93-8f28-f2788f6bff8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:33.990225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-1e29a3bb-f652-4d93-8f28-f2788f6bff8a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 13:22:33.990225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:33.990225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:33.990225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:33.990225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:33.990225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:33.990225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:33.990225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:33.990225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:33.990225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:33.990225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:33.990225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:33.990225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:33.990225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:33.990225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:33.990225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:33.991410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:33.991410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:33.991410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:33.991410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:33.991410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:33.991410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:33.991410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:33.991410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:33.991410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:33.991410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:33.991410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:33.991410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:33.991410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:33.991410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:33.991410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:33.991410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:33.992214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:33.992214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:33.992214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:33.992214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:33.992214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:33.992214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:33.992214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:33.992214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:33.992214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:33.992214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:33.992214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 13:22:33.992214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:22:33.994839 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 48/96] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:22:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:22:33.996754 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc534c-3ac1-4000-b859-9291c206a553 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:33.997262 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc534c-3ac1-4000-b859-9291c206a553 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:33.997764 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc534c-3ac1-4000-b859-9291c206a553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:34.017429 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-d6cc534c-3ac1-4000-b859-9291c206a553 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:34.018542 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-d6cc534c-3ac1-4000-b859-9291c206a553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:34.086837 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 49/97] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:34.087486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc534c-3ac1-4000-b859-9291c206a553 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:34.092689 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae456ea-d1c1-4887-87e1-7be66fb175ce None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:34.117579 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:34.134106 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af6b55-72fd-4a02-ba8f-db9f3bf2b5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:34.134748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af6b55-72fd-4a02-ba8f-db9f3bf2b5fa None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:34.135031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af6b55-72fd-4a02-ba8f-db9f3bf2b5fa None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:34.135309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af6b55-72fd-4a02-ba8f-db9f3bf2b5fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:34.149368 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af6b55-72fd-4a02-ba8f-db9f3bf2b5fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:34.155839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-e2af6b55-72fd-4a02-ba8f-db9f3bf2b5fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:22:34.155839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:34.155839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:34.155839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:34.155839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:34.155839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:34.155839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:34.155839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:34.155839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:34.155839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:34.155839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:34.155839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:34.155839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:34.155839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:34.155839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:34.155839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:34.156678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:34.156678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:34.156678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:34.156678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:34.156678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:34.156678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:34.156678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:34.156678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:34.156678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:34.156678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:34.156678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:34.156678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:34.156678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:34.156678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:34.156678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:34.156678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:34.161776 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:34.161776 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:34.161776 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:34.161776 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:34.161776 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:34.161776 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:34.161776 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:34.161776 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:34.161776 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:34.161776 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:34.161776 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:22:34.161776 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:22:34.161776 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 49/98] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:22:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:22:34.161776 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af6b55-72fd-4a02-ba8f-db9f3bf2b5fa None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:34.213190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc534c-3ac1-4000-b859-9291c206a553 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:34.239086 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:34.249879 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f8203-f69a-4ed9-a345-089ee98b61f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:34.250744 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f8203-f69a-4ed9-a345-089ee98b61f3 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:34.251175 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f8203-f69a-4ed9-a345-089ee98b61f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:34.251596 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f8203-f69a-4ed9-a345-089ee98b61f3 None admin] PATH_INFO: `/v3/users/b0595b7096954e6a802224bb45e3f3cf` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:34.253069 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0f8203-f69a-4ed9-a345-089ee98b61f3 None admin] RBAC: Authorizing `identity:get_user(user_id=b0595b7096954e6a802224bb45e3f3cf)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:34.254743 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0f8203-f69a-4ed9-a345-089ee98b61f3 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:34.257178 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 50/99] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:34 2022] GET /identity/v3/users/b0595b7096954e6a802224bb45e3f3cf => generated 247 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:34.258626 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4efb3b8-c10e-400e-9ccb-2c8167386752 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:34.259126 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4efb3b8-c10e-400e-9ccb-2c8167386752 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:34.259354 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4efb3b8-c10e-400e-9ccb-2c8167386752 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:34.260713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 51/100] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:34.266547 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4efb3b8-c10e-400e-9ccb-2c8167386752 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:34.285931 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af6b55-72fd-4a02-ba8f-db9f3bf2b5fa None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:34.326065 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53755da-fa0f-4590-96df-6e405c16119b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:34.327051 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53755da-fa0f-4590-96df-6e405c16119b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:34.327573 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53755da-fa0f-4590-96df-6e405c16119b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:34.328105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53755da-fa0f-4590-96df-6e405c16119b None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:34.329172 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53755da-fa0f-4590-96df-6e405c16119b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:34.329484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53755da-fa0f-4590-96df-6e405c16119b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:34.331080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53755da-fa0f-4590-96df-6e405c16119b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:34.344523 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 50/101] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 20 13:22:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:34.345527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53755da-fa0f-4590-96df-6e405c16119b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:34.368823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:34.380821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4efb3b8-c10e-400e-9ccb-2c8167386752 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:34.419970 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9c9f0-8ada-4094-8f4e-6e848c309b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:34.420592 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9c9f0-8ada-4094-8f4e-6e848c309b05 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:34.420997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9c9f0-8ada-4094-8f4e-6e848c309b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:34.421199 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9c9f0-8ada-4094-8f4e-6e848c309b05 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:34.421870 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c9c9f0-8ada-4094-8f4e-6e848c309b05 None admin] RBAC: Adding query filter params (name=member) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:34.422183 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c9c9f0-8ada-4094-8f4e-6e848c309b05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:34.423591 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c9c9f0-8ada-4094-8f4e-6e848c309b05 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:34.433556 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 52/102] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 20 13:22:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:34.434622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dddf7-71fe-4f5f-9d05-4c8a06b63bbf None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:34.434915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dddf7-71fe-4f5f-9d05-4c8a06b63bbf None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:34.435195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dddf7-71fe-4f5f-9d05-4c8a06b63bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:34.453096 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-207dddf7-71fe-4f5f-9d05-4c8a06b63bbf None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:34.454274 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-207dddf7-71fe-4f5f-9d05-4c8a06b63bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:34.470399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53755da-fa0f-4590-96df-6e405c16119b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:34.503029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:34.512498 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4462617f-5e81-407d-ae63-beaf6d690fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:34.513390 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462617f-5e81-407d-ae63-beaf6d690fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:34.513670 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462617f-5e81-407d-ae63-beaf6d690fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:34.513943 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462617f-5e81-407d-ae63-beaf6d690fa0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:34.522856 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 53/103] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:34.523699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207dddf7-71fe-4f5f-9d05-4c8a06b63bbf None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:34.585623 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4462617f-5e81-407d-ae63-beaf6d690fa0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:34.587103 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4462617f-5e81-407d-ae63-beaf6d690fa0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:34.603268 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:34.622615 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207dddf7-71fe-4f5f-9d05-4c8a06b63bbf None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:34.655663 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c983db9-115f-4bed-89d7-1855ac7f6a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:34.656296 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c983db9-115f-4bed-89d7-1855ac7f6a95 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:34.656577 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c983db9-115f-4bed-89d7-1855ac7f6a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:34.656849 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c983db9-115f-4bed-89d7-1855ac7f6a95 None admin] PATH_INFO: `/v3/users/4b98d5861c2a46e783bfbe3e9717661d` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:34.670383 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c983db9-115f-4bed-89d7-1855ac7f6a95 None admin] RBAC: Authorizing `identity:get_user(user_id=4b98d5861c2a46e783bfbe3e9717661d)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:34.671831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c983db9-115f-4bed-89d7-1855ac7f6a95 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:34.673652 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 54/104] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:34 2022] GET /identity/v3/users/4b98d5861c2a46e783bfbe3e9717661d => generated 289 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:34.674239 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c983db9-115f-4bed-89d7-1855ac7f6a95 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:34.696824 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 51/105] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:22:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 13:22:34.697978 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4462617f-5e81-407d-ae63-beaf6d690fa0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:34.727076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:34.779262 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c983db9-115f-4bed-89d7-1855ac7f6a95 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:34.805767 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4462617f-5e81-407d-ae63-beaf6d690fa0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:34.812788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72400ae2-4fab-4472-be40-df9ddfebbc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:34.813452 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72400ae2-4fab-4472-be40-df9ddfebbc5b None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:34.813729 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72400ae2-4fab-4472-be40-df9ddfebbc5b None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:34.814001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72400ae2-4fab-4472-be40-df9ddfebbc5b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:34.821309 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:34.826039 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72400ae2-4fab-4472-be40-df9ddfebbc5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:34.830881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-72400ae2-4fab-4472-be40-df9ddfebbc5b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 13:22:34.830881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:34.830881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:34.830881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:34.830881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:34.830881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:34.830881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:34.830881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:34.830881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:34.830881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:34.830881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:34.830881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:34.830881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:34.830881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:34.830881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:34.830881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:34.831915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:34.831915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:34.831915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:34.831915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:34.831915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:34.831915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:34.831915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:34.831915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:34.831915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:34.831915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:34.831915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:34.831915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:34.831915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:34.831915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:34.831915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:34.831915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:34.832902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:34.832902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:34.832902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:34.832902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:34.832902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:34.832902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:34.832902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:34.832902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:34.832902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:34.832902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:34.832902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 13:22:34.832902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:22:34.832902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 55/106] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 20 13:22:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:22:34.833688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71082e35-93bd-4bf4-ba5b-e275cbe65596 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:34.833863 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71082e35-93bd-4bf4-ba5b-e275cbe65596 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:34.834648 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71082e35-93bd-4bf4-ba5b-e275cbe65596 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:34.835951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 56/107] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:34.836587 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71082e35-93bd-4bf4-ba5b-e275cbe65596 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:34.844789 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e261c-88d9-4a8e-8d39-fcf82338edb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:34.845447 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e261c-88d9-4a8e-8d39-fcf82338edb6 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:34.845732 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e261c-88d9-4a8e-8d39-fcf82338edb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:34.846016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e261c-88d9-4a8e-8d39-fcf82338edb6 None admin] PATH_INFO: `/v3/users/b7bc816de9e04a01adb1bc809287ea78` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:34.847113 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e261c-88d9-4a8e-8d39-fcf82338edb6 None admin] RBAC: Authorizing `identity:get_user(user_id=b7bc816de9e04a01adb1bc809287ea78)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:34.848544 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e261c-88d9-4a8e-8d39-fcf82338edb6 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:34.850773 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 52/108] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:34 2022] GET /identity/v3/users/b7bc816de9e04a01adb1bc809287ea78 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:34.851533 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e261c-88d9-4a8e-8d39-fcf82338edb6 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:34.937444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71082e35-93bd-4bf4-ba5b-e275cbe65596 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:34.942647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:34.948669 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e261c-88d9-4a8e-8d39-fcf82338edb6 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:34.970714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7437c1-ca91-49e8-ba44-57e8dabc517b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:34.971469 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7437c1-ca91-49e8-ba44-57e8dabc517b None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:34.971755 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7437c1-ca91-49e8-ba44-57e8dabc517b None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:34.972032 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7437c1-ca91-49e8-ba44-57e8dabc517b None admin] PATH_INFO: `/v3/projects/2a7d45346e944c6089c645b9c6958126` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:34.973205 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7437c1-ca91-49e8-ba44-57e8dabc517b None admin] RBAC: Authorizing `identity:get_project(project_id=2a7d45346e944c6089c645b9c6958126)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:34.974744 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7437c1-ca91-49e8-ba44-57e8dabc517b None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:34.976495 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 57/109] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:22:34 2022] GET /identity/v3/projects/2a7d45346e944c6089c645b9c6958126 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:34.977282 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7437c1-ca91-49e8-ba44-57e8dabc517b None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:34.987208 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b650bc-2e26-40f4-a8d4-1a66d5095e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:34.987836 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b650bc-2e26-40f4-a8d4-1a66d5095e3b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:34.988120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b650bc-2e26-40f4-a8d4-1a66d5095e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:34.988391 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b650bc-2e26-40f4-a8d4-1a66d5095e3b None admin] PATH_INFO: `/v3/projects/2a7d45346e944c6089c645b9c6958126` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:34.989511 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b650bc-2e26-40f4-a8d4-1a66d5095e3b None admin] RBAC: Authorizing `identity:get_project(project_id=2a7d45346e944c6089c645b9c6958126)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:34.991655 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b650bc-2e26-40f4-a8d4-1a66d5095e3b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:34.997941 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 53/110] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:22:34 2022] GET /identity/v3/projects/2a7d45346e944c6089c645b9c6958126 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:34.997941 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b650bc-2e26-40f4-a8d4-1a66d5095e3b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:35.042840 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:35.079573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7437c1-ca91-49e8-ba44-57e8dabc517b None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:35.105584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b650bc-2e26-40f4-a8d4-1a66d5095e3b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:35.109584 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee74e79-f7ec-4513-81d2-5b07235d369c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:35.110221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee74e79-f7ec-4513-81d2-5b07235d369c None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:35.110509 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee74e79-f7ec-4513-81d2-5b07235d369c None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:35.110790 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee74e79-f7ec-4513-81d2-5b07235d369c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:35.111900 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee74e79-f7ec-4513-81d2-5b07235d369c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:35.113465 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee74e79-f7ec-4513-81d2-5b07235d369c None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:35.115055 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 58/111] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:22:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:35.116729 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7769e8-0bb3-42ea-b80a-d80813656f00 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:35.117043 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7769e8-0bb3-42ea-b80a-d80813656f00 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:35.117322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7769e8-0bb3-42ea-b80a-d80813656f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:35.134737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-4c7769e8-0bb3-42ea-b80a-d80813656f00 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:35.135941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-4c7769e8-0bb3-42ea-b80a-d80813656f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:35.140948 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8d834-b179-4dc6-9566-412ab232ecc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:35.141550 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8d834-b179-4dc6-9566-412ab232ecc3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:35.141829 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8d834-b179-4dc6-9566-412ab232ecc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:35.142098 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8d834-b179-4dc6-9566-412ab232ecc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:35.142756 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e8d834-b179-4dc6-9566-412ab232ecc3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:35.143057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e8d834-b179-4dc6-9566-412ab232ecc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:35.144421 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e8d834-b179-4dc6-9566-412ab232ecc3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:35.152834 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 54/112] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:22:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:35.156959 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8d834-b179-4dc6-9566-412ab232ecc3 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:35.183710 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:35.202274 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 59/113] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:35.203108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7769e8-0bb3-42ea-b80a-d80813656f00 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:35.260049 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8d834-b179-4dc6-9566-412ab232ecc3 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:35.276263 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:35.297184 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232f529-7a05-4c79-bb07-7d40954a1169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:35.297796 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232f529-7a05-4c79-bb07-7d40954a1169 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:35.298090 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232f529-7a05-4c79-bb07-7d40954a1169 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:35.298361 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232f529-7a05-4c79-bb07-7d40954a1169 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:35.313325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7769e8-0bb3-42ea-b80a-d80813656f00 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:35.347503 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa4dca-2d65-4110-b7c5-42fcb9dd5008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:35.348168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa4dca-2d65-4110-b7c5-42fcb9dd5008 None admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:35.348454 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa4dca-2d65-4110-b7c5-42fcb9dd5008 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:35.348732 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa4dca-2d65-4110-b7c5-42fcb9dd5008 None admin] PATH_INFO: `/v3/projects/2a7d45346e944c6089c645b9c6958126/users/4b98d5861c2a46e783bfbe3e9717661d/roles/83ca567af06444b3b95843cb52d61698` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:35.351228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fa4dca-2d65-4110-b7c5-42fcb9dd5008 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a7d45346e944c6089c645b9c6958126, user_id=4b98d5861c2a46e783bfbe3e9717661d, role_id=83ca567af06444b3b95843cb52d61698)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:35.352686 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fa4dca-2d65-4110-b7c5-42fcb9dd5008 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:35.365778 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 60/114] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:22:35 2022] PUT /identity/v3/projects/2a7d45346e944c6089c645b9c6958126/users/4b98d5861c2a46e783bfbe3e9717661d/roles/83ca567af06444b3b95843cb52d61698 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:22:35.366923 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa4dca-2d65-4110-b7c5-42fcb9dd5008 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:35.384095 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0232f529-7a05-4c79-bb07-7d40954a1169 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:35.385832 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0232f529-7a05-4c79-bb07-7d40954a1169 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:35.421366 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:35.488174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa4dca-2d65-4110-b7c5-42fcb9dd5008 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:35.523470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:35.525959 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 55/115] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:22:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 13:22:35.526825 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232f529-7a05-4c79-bb07-7d40954a1169 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:35.534577 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1fe927-803a-4f5a-897f-cc25f6900dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:35.535208 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1fe927-803a-4f5a-897f-cc25f6900dfe None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:35.535496 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1fe927-803a-4f5a-897f-cc25f6900dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:35.535770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1fe927-803a-4f5a-897f-cc25f6900dfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:35.537134 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1fe927-803a-4f5a-897f-cc25f6900dfe None admin] RBAC: Adding query filter params (role.id=2bcf35e1bc5f4634b0aa6411fad2f43f, scope.project.id=2a7d45346e944c6089c645b9c6958126, user.id=b0595b7096954e6a802224bb45e3f3cf) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:35.538386 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1fe927-803a-4f5a-897f-cc25f6900dfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:35.540034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1fe927-803a-4f5a-897f-cc25f6900dfe None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:35.548644 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 61/116] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:22:35 2022] GET /identity/v3/role_assignments?user.id=b0595b7096954e6a802224bb45e3f3cf&scope.project.id=2a7d45346e944c6089c645b9c6958126&role.id=2bcf35e1bc5f4634b0aa6411fad2f43f => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:35.549375 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1fe927-803a-4f5a-897f-cc25f6900dfe None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:35.659899 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232f529-7a05-4c79-bb07-7d40954a1169 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:35.685182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1fe927-803a-4f5a-897f-cc25f6900dfe None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:35.693203 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:35.706720 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce04da0-8456-4979-971f-126c694ee6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:35.707764 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce04da0-8456-4979-971f-126c694ee6bc None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:35.708258 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce04da0-8456-4979-971f-126c694ee6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:35.708728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce04da0-8456-4979-971f-126c694ee6bc None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:35.710420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce04da0-8456-4979-971f-126c694ee6bc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:35.712236 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce04da0-8456-4979-971f-126c694ee6bc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:35.731117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d7893-f998-464b-bdfc-e88969ef5e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:35.732036 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d7893-f998-464b-bdfc-e88969ef5e31 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:35.732328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d7893-f998-464b-bdfc-e88969ef5e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:35.732611 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d7893-f998-464b-bdfc-e88969ef5e31 None admin] PATH_INFO: `/v3/users/b7bc816de9e04a01adb1bc809287ea78` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:35.734313 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220d7893-f998-464b-bdfc-e88969ef5e31 None admin] RBAC: Authorizing `identity:get_user(user_id=b7bc816de9e04a01adb1bc809287ea78)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:35.736272 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220d7893-f998-464b-bdfc-e88969ef5e31 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:35.739589 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 62/117] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:35 2022] GET /identity/v3/users/b7bc816de9e04a01adb1bc809287ea78 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:35.739589 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d7893-f998-464b-bdfc-e88969ef5e31 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:35.750820 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 56/118] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 20 13:22:35 2022] POST /identity/v3/projects => generated 302 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:22:35.752033 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce04da0-8456-4979-971f-126c694ee6bc None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:35.812480 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:35.875271 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d7893-f998-464b-bdfc-e88969ef5e31 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:35.899477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce04da0-8456-4979-971f-126c694ee6bc None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:35.937094 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f140b-74e4-482c-a96e-cbcc32e11a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:35.937777 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f140b-74e4-482c-a96e-cbcc32e11a66 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:35.938169 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f140b-74e4-482c-a96e-cbcc32e11a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:35.938451 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f140b-74e4-482c-a96e-cbcc32e11a66 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:35.943120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b174585-3a61-4f28-bfa0-9b436082e930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:35.944310 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b174585-3a61-4f28-bfa0-9b436082e930 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:35.944637 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b174585-3a61-4f28-bfa0-9b436082e930 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:35.945014 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b174585-3a61-4f28-bfa0-9b436082e930 None admin] PATH_INFO: `/v3/projects/2a7d45346e944c6089c645b9c6958126` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:35.946564 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b174585-3a61-4f28-bfa0-9b436082e930 None admin] RBAC: Authorizing `identity:get_project(project_id=2a7d45346e944c6089c645b9c6958126)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:35.948057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b174585-3a61-4f28-bfa0-9b436082e930 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:35.950253 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 57/119] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:22:35 2022] GET /identity/v3/projects/2a7d45346e944c6089c645b9c6958126 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:35.951326 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b174585-3a61-4f28-bfa0-9b436082e930 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:35.953413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f140b-74e4-482c-a96e-cbcc32e11a66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:35.959153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-e69f140b-74e4-482c-a96e-cbcc32e11a66 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:22:35.959153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:35.959153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:35.959153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:35.959153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:35.959153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:35.959153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:35.959153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:35.959153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:35.959153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:35.959153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:35.959153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:35.959153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:35.959153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:35.959153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:35.959153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:35.960118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:35.960118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:35.960118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:35.960118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:35.960118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:35.960118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:35.960118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:35.960118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:35.960118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:35.960118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:35.960118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:35.960118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:35.960118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:35.960118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:35.960118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:35.960118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:35.960847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:35.960847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:35.960847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:35.960847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:35.960847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:35.960847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:35.960847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:35.960847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:35.960847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:35.960847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:35.960847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:22:35.960847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:22:35.961435 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 63/120] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:22:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:22:35.962262 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f140b-74e4-482c-a96e-cbcc32e11a66 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:36.009017 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:36.077974 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b174585-3a61-4f28-bfa0-9b436082e930 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:36.104912 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f140b-74e4-482c-a96e-cbcc32e11a66 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:36.122845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363cee8-7374-494c-8fe3-c2c1875a7a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:36.123580 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363cee8-7374-494c-8fe3-c2c1875a7a7b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:36.123868 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363cee8-7374-494c-8fe3-c2c1875a7a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:36.124149 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363cee8-7374-494c-8fe3-c2c1875a7a7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:36.124644 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:36.142761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2773894-31df-49d8-94da-6ef50e5739b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:36.143479 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2773894-31df-49d8-94da-6ef50e5739b3 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:36.143835 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2773894-31df-49d8-94da-6ef50e5739b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:36.144185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2773894-31df-49d8-94da-6ef50e5739b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:36.145016 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2773894-31df-49d8-94da-6ef50e5739b3 None admin] RBAC: Adding query filter params (role.id=5c47f5b95d6f47a78adf8f8e91562e4b, scope.project.id=2a7d45346e944c6089c645b9c6958126, user.id=b7bc816de9e04a01adb1bc809287ea78) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:36.145395 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2773894-31df-49d8-94da-6ef50e5739b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:36.147015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2773894-31df-49d8-94da-6ef50e5739b3 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:36.153048 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 64/121] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:22:35 2022] GET /identity/v3/role_assignments?user.id=b7bc816de9e04a01adb1bc809287ea78&scope.project.id=2a7d45346e944c6089c645b9c6958126&role.id=5c47f5b95d6f47a78adf8f8e91562e4b => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:36.153958 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2773894-31df-49d8-94da-6ef50e5739b3 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:36.207204 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2363cee8-7374-494c-8fe3-c2c1875a7a7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:36.208690 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2363cee8-7374-494c-8fe3-c2c1875a7a7b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:36.267266 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:36.297224 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2773894-31df-49d8-94da-6ef50e5739b3 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:36.335054 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 58/122] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:22:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 13:22:36.341297 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363cee8-7374-494c-8fe3-c2c1875a7a7b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:36.378163 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91ffa21-c405-4990-9759-1802b6b83abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:36.378788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ffa21-c405-4990-9759-1802b6b83abe None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:36.379073 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ffa21-c405-4990-9759-1802b6b83abe None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:36.379347 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ffa21-c405-4990-9759-1802b6b83abe None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:36.380019 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91ffa21-c405-4990-9759-1802b6b83abe None admin] RBAC: Adding query filter params (name=member) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:36.380326 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91ffa21-c405-4990-9759-1802b6b83abe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:36.381932 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91ffa21-c405-4990-9759-1802b6b83abe None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:36.403433 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 65/123] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 20 13:22:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:36.409007 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91ffa21-c405-4990-9759-1802b6b83abe None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:36.458027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363cee8-7374-494c-8fe3-c2c1875a7a7b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:36.486095 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:36.500770 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e6475-bf01-4747-a87d-7bb03947ed17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:36.501470 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e6475-bf01-4747-a87d-7bb03947ed17 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:36.501761 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e6475-bf01-4747-a87d-7bb03947ed17 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:36.502045 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e6475-bf01-4747-a87d-7bb03947ed17 None admin] PATH_INFO: `/v3/projects/2a7d45346e944c6089c645b9c6958126` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:36.503191 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e6475-bf01-4747-a87d-7bb03947ed17 None admin] RBAC: Authorizing `identity:get_project(project_id=2a7d45346e944c6089c645b9c6958126)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:36.504656 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e6475-bf01-4747-a87d-7bb03947ed17 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:36.507849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 59/124] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:22:36 2022] GET /identity/v3/projects/2a7d45346e944c6089c645b9c6958126 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:22:36.513216 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e6475-bf01-4747-a87d-7bb03947ed17 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:36.529526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:36.557069 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91ffa21-c405-4990-9759-1802b6b83abe None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:36.594983 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4c7ac-a816-4a60-a4ba-abe98913e209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:36.596020 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4c7ac-a816-4a60-a4ba-abe98913e209 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:36.596497 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4c7ac-a816-4a60-a4ba-abe98913e209 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:36.597001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4c7ac-a816-4a60-a4ba-abe98913e209 None admin] PATH_INFO: `/v3/users/b7bc816de9e04a01adb1bc809287ea78` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:36.598411 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4c7ac-a816-4a60-a4ba-abe98913e209 None admin] RBAC: Authorizing `identity:get_user(user_id=b7bc816de9e04a01adb1bc809287ea78)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:36.600242 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4c7ac-a816-4a60-a4ba-abe98913e209 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:36.602640 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 66/125] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:36 2022] GET /identity/v3/users/b7bc816de9e04a01adb1bc809287ea78 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:22:36.608552 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4c7ac-a816-4a60-a4ba-abe98913e209 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:36.617880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e6475-bf01-4747-a87d-7bb03947ed17 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:36.660985 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e4ed0-377e-41df-87f5-f8d6d47d208b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:36.661661 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e4ed0-377e-41df-87f5-f8d6d47d208b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:36.661951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e4ed0-377e-41df-87f5-f8d6d47d208b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:36.662330 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e4ed0-377e-41df-87f5-f8d6d47d208b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:36.663135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e4ed0-377e-41df-87f5-f8d6d47d208b None admin] RBAC: Adding query filter params (role.id=b0ccf031da9e4a7492c567b607ceb21f, scope.project.id=2a7d45346e944c6089c645b9c6958126, user.id=b7bc816de9e04a01adb1bc809287ea78) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:36.663828 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e4ed0-377e-41df-87f5-f8d6d47d208b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:36.665702 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e4ed0-377e-41df-87f5-f8d6d47d208b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:36.672835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 60/126] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:22:36 2022] GET /identity/v3/role_assignments?user.id=b7bc816de9e04a01adb1bc809287ea78&scope.project.id=2a7d45346e944c6089c645b9c6958126&role.id=b0ccf031da9e4a7492c567b607ceb21f => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:36.742149 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4c7ac-a816-4a60-a4ba-abe98913e209 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:36.758390 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:36.783679 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda7cd7-f8b0-4d10-ad9f-db67ca48e110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:36.784725 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda7cd7-f8b0-4d10-ad9f-db67ca48e110 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:36.785292 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda7cd7-f8b0-4d10-ad9f-db67ca48e110 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:36.785569 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda7cd7-f8b0-4d10-ad9f-db67ca48e110 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:36.948056 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda7cd7-f8b0-4d10-ad9f-db67ca48e110 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:36.949616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda7cd7-f8b0-4d10-ad9f-db67ca48e110 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:37.059046 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 67/127] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:22:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 13:22:37.065268 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e4ed0-377e-41df-87f5-f8d6d47d208b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:37.147206 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:37.175844 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e4ed0-377e-41df-87f5-f8d6d47d208b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:37.214971 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988acad9-ac05-4a6d-8bde-2ea65ce1cac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:37.215941 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988acad9-ac05-4a6d-8bde-2ea65ce1cac9 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:37.216699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988acad9-ac05-4a6d-8bde-2ea65ce1cac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:37.218056 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988acad9-ac05-4a6d-8bde-2ea65ce1cac9 None admin] PATH_INFO: `/v3/projects/21788bc08f594f52839376f2dd54ddba` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:37.219722 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988acad9-ac05-4a6d-8bde-2ea65ce1cac9 None admin] RBAC: Authorizing `identity:get_project(project_id=21788bc08f594f52839376f2dd54ddba)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:37.221830 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988acad9-ac05-4a6d-8bde-2ea65ce1cac9 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:37.224985 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 61/128] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:22:37 2022] GET /identity/v3/projects/21788bc08f594f52839376f2dd54ddba => generated 312 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:37.233337 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda7cd7-f8b0-4d10-ad9f-db67ca48e110 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:37.249219 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:37.338146 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda7cd7-f8b0-4d10-ad9f-db67ca48e110 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:37.373200 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c2ae3-ebb0-4754-b967-6c70026568df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:37.373814 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c2ae3-ebb0-4754-b967-6c70026568df None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:37.374095 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c2ae3-ebb0-4754-b967-6c70026568df None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:37.374370 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c2ae3-ebb0-4754-b967-6c70026568df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:37.375095 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c2ae3-ebb0-4754-b967-6c70026568df None admin] RBAC: Adding query filter params (role.id=5c47f5b95d6f47a78adf8f8e91562e4b, scope.project.id=21788bc08f594f52839376f2dd54ddba, user.id=b7bc816de9e04a01adb1bc809287ea78) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:37.375411 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c2ae3-ebb0-4754-b967-6c70026568df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:37.376897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c2ae3-ebb0-4754-b967-6c70026568df None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:37.383475 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 68/129] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:22:37 2022] GET /identity/v3/role_assignments?user.id=b7bc816de9e04a01adb1bc809287ea78&scope.project.id=21788bc08f594f52839376f2dd54ddba&role.id=5c47f5b95d6f47a78adf8f8e91562e4b => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:37.816518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59878f1f-3591-41fe-a965-7d5214743184 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:37.816806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59878f1f-3591-41fe-a965-7d5214743184 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:37.817114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59878f1f-3591-41fe-a965-7d5214743184 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:37.818511 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 62/130] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:37.823868 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55424c1e-6113-4963-b936-3aa87299b44b None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:37.824226 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55424c1e-6113-4963-b936-3aa87299b44b None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:37.824512 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55424c1e-6113-4963-b936-3aa87299b44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:37.842779 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-55424c1e-6113-4963-b936-3aa87299b44b None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:37.843941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-55424c1e-6113-4963-b936-3aa87299b44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:37.909300 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 69/131] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:38.175770 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59878f1f-3591-41fe-a965-7d5214743184 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:38.263494 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9d758-4736-444c-b2e8-d0fc2753a4a3 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:38.264016 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9d758-4736-444c-b2e8-d0fc2753a4a3 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:38.264016 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9d758-4736-444c-b2e8-d0fc2753a4a3 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:38.265572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 70/132] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:38.270058 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9e4c8-79db-4a05-8656-1adf871cbe13 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:38.270382 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9e4c8-79db-4a05-8656-1adf871cbe13 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:38.270658 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9e4c8-79db-4a05-8656-1adf871cbe13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:38.275341 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59878f1f-3591-41fe-a965-7d5214743184 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:38.283199 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:38.289911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-07f9e4c8-79db-4a05-8656-1adf871cbe13 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:38.291089 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-07f9e4c8-79db-4a05-8656-1adf871cbe13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:38.308852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ab6ee-dd27-4cb9-91c1-371933974788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:38.309513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ab6ee-dd27-4cb9-91c1-371933974788 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:38.309795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ab6ee-dd27-4cb9-91c1-371933974788 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:38.310071 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ab6ee-dd27-4cb9-91c1-371933974788 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:38.320431 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ab6ee-dd27-4cb9-91c1-371933974788 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:38.325758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-079ab6ee-dd27-4cb9-91c1-371933974788 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 13:22:38.325758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:38.325758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:38.325758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:38.325758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:38.325758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:38.325758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:38.325758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:38.325758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:38.325758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:38.325758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:38.325758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:38.325758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:38.325758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:38.325758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:38.325758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:38.326731 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:38.326731 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:38.326731 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:38.326731 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:38.326731 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:38.326731 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:38.326731 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:38.326731 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:38.326731 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:38.326731 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:38.326731 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:38.326731 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:38.326731 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:38.326731 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:38.326731 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:38.326731 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:38.327530 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:38.327530 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:38.327530 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:38.327530 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:38.327530 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:38.327530 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:38.327530 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:38.327530 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:38.327530 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:38.327530 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:38.327530 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 13:22:38.327530 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:22:38.327530 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 63/133] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:22:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:22:38.332671 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ab6ee-dd27-4cb9-91c1-371933974788 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:38.341889 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:38.356359 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 71/134] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 13:22:38.429506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ab6ee-dd27-4cb9-91c1-371933974788 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:38.461291 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95579ba-c5e9-4655-99f9-5cbb98c5e3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:38.461892 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95579ba-c5e9-4655-99f9-5cbb98c5e3ea None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:38.462172 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95579ba-c5e9-4655-99f9-5cbb98c5e3ea None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:38.462444 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95579ba-c5e9-4655-99f9-5cbb98c5e3ea None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:38.463105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95579ba-c5e9-4655-99f9-5cbb98c5e3ea None admin] RBAC: Adding query filter params (name=reader) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:38.463410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95579ba-c5e9-4655-99f9-5cbb98c5e3ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:38.464789 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95579ba-c5e9-4655-99f9-5cbb98c5e3ea None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:38.472471 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 64/135] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 20 13:22:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:22:38.478794 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9e4c8-79db-4a05-8656-1adf871cbe13 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:38.576272 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9e4c8-79db-4a05-8656-1adf871cbe13 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:38.608626 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f93fa-94c2-49cd-9bab-5ebb3254772d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:38.609269 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f93fa-94c2-49cd-9bab-5ebb3254772d None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:38.609551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f93fa-94c2-49cd-9bab-5ebb3254772d None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:38.609826 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f93fa-94c2-49cd-9bab-5ebb3254772d None admin] PATH_INFO: `/v3/users/4b98d5861c2a46e783bfbe3e9717661d` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:38.610939 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f93fa-94c2-49cd-9bab-5ebb3254772d None admin] RBAC: Authorizing `identity:get_user(user_id=4b98d5861c2a46e783bfbe3e9717661d)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:38.612379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f93fa-94c2-49cd-9bab-5ebb3254772d None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:38.614140 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 72/136] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:38 2022] GET /identity/v3/users/4b98d5861c2a46e783bfbe3e9717661d => generated 289 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:38.619683 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95579ba-c5e9-4655-99f9-5cbb98c5e3ea None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:38.624121 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f93fa-94c2-49cd-9bab-5ebb3254772d None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:38.628069 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:38.688520 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:38.722928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95579ba-c5e9-4655-99f9-5cbb98c5e3ea None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:38.723965 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f93fa-94c2-49cd-9bab-5ebb3254772d None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:38.756606 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8aebf-17ea-4d9e-a7c6-5e52540918d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:38.757239 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8aebf-17ea-4d9e-a7c6-5e52540918d4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:38.757518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8aebf-17ea-4d9e-a7c6-5e52540918d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:38.757788 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8aebf-17ea-4d9e-a7c6-5e52540918d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:38.758701 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da72e8-b727-4bc1-bd77-10135d5e8828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:38.759395 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da72e8-b727-4bc1-bd77-10135d5e8828 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:38.759743 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da72e8-b727-4bc1-bd77-10135d5e8828 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:38.760207 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da72e8-b727-4bc1-bd77-10135d5e8828 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:38.761231 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da72e8-b727-4bc1-bd77-10135d5e8828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:38.762933 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da72e8-b727-4bc1-bd77-10135d5e8828 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:38.764520 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 73/137] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:22:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:22:38.769877 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da72e8-b727-4bc1-bd77-10135d5e8828 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:38.825718 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e8aebf-17ea-4d9e-a7c6-5e52540918d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:38.827190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e8aebf-17ea-4d9e-a7c6-5e52540918d4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:38.839521 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:38.870039 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da72e8-b727-4bc1-bd77-10135d5e8828 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:38.890724 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:38.904199 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043bc97-ece3-4631-b23f-0a1bafdb92b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:38.904805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043bc97-ece3-4631-b23f-0a1bafdb92b2 None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:38.905122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043bc97-ece3-4631-b23f-0a1bafdb92b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:38.905402 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043bc97-ece3-4631-b23f-0a1bafdb92b2 None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:38.906060 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043bc97-ece3-4631-b23f-0a1bafdb92b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:38.907518 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043bc97-ece3-4631-b23f-0a1bafdb92b2 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:38.926422 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 65/138] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:22:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 13:22:38.932899 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 74/139] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 20 13:22:38 2022] POST /identity/v3/users => generated 283 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:22:38.933085 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8aebf-17ea-4d9e-a7c6-5e52540918d4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:39.036121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8aebf-17ea-4d9e-a7c6-5e52540918d4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:39.072998 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b2f70-5827-496c-bb2b-298b80a1d84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:39.073654 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b2f70-5827-496c-bb2b-298b80a1d84d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:39.073938 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b2f70-5827-496c-bb2b-298b80a1d84d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:39.074216 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b2f70-5827-496c-bb2b-298b80a1d84d None admin] PATH_INFO: `/v3/projects/2a7d45346e944c6089c645b9c6958126` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:39.075371 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02b2f70-5827-496c-bb2b-298b80a1d84d None admin] RBAC: Authorizing `identity:get_project(project_id=2a7d45346e944c6089c645b9c6958126)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:39.076833 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02b2f70-5827-496c-bb2b-298b80a1d84d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:39.078529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 66/140] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:22:38 2022] GET /identity/v3/projects/2a7d45346e944c6089c645b9c6958126 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:39.085256 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043bc97-ece3-4631-b23f-0a1bafdb92b2 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:39.187345 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043bc97-ece3-4631-b23f-0a1bafdb92b2 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:39.214272 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:39.225036 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec58fe9-8ca6-4de6-8ee0-9415953a1e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:39.225036 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec58fe9-8ca6-4de6-8ee0-9415953a1e9d None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:39.225036 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec58fe9-8ca6-4de6-8ee0-9415953a1e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:39.225413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec58fe9-8ca6-4de6-8ee0-9415953a1e9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:39.226226 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec58fe9-8ca6-4de6-8ee0-9415953a1e9d None admin] RBAC: Adding query filter params (role.id=83ca567af06444b3b95843cb52d61698, scope.project.id=2a7d45346e944c6089c645b9c6958126, user.id=4b98d5861c2a46e783bfbe3e9717661d) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:39.226537 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec58fe9-8ca6-4de6-8ee0-9415953a1e9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:39.227987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec58fe9-8ca6-4de6-8ee0-9415953a1e9d None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:39.234983 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 75/141] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:22:39 2022] GET /identity/v3/role_assignments?user.id=4b98d5861c2a46e783bfbe3e9717661d&scope.project.id=2a7d45346e944c6089c645b9c6958126&role.id=83ca567af06444b3b95843cb52d61698 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:41.206343 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0a5e1-825c-4ebd-9173-e6c55cd49708 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:41.206880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0a5e1-825c-4ebd-9173-e6c55cd49708 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:41.207888 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0a5e1-825c-4ebd-9173-e6c55cd49708 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:41.209956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 67/142] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:41.215234 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d486308-33a5-4b48-87cd-29d4aec4466f None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:41.215696 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d486308-33a5-4b48-87cd-29d4aec4466f None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:41.215922 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d486308-33a5-4b48-87cd-29d4aec4466f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:41.237526 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-4d486308-33a5-4b48-87cd-29d4aec4466f None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:41.238828 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-4d486308-33a5-4b48-87cd-29d4aec4466f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:41.316009 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 76/143] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:41.371983 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ceb94-373d-4dd0-9fb8-2d289dd560b8 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:41.372285 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ceb94-373d-4dd0-9fb8-2d289dd560b8 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:41.372553 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ceb94-373d-4dd0-9fb8-2d289dd560b8 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:41.374645 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 68/144] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:41.380055 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848e3c5-f853-4b3a-a952-812ec1cc40d3 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:41.380501 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848e3c5-f853-4b3a-a952-812ec1cc40d3 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:41.381052 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848e3c5-f853-4b3a-a952-812ec1cc40d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:41.402998 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-a848e3c5-f853-4b3a-a952-812ec1cc40d3 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:41.404183 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-a848e3c5-f853-4b3a-a952-812ec1cc40d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:41.470193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88129e47-7da8-4f18-81fb-6e2e4546ca8f None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:41.470453 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88129e47-7da8-4f18-81fb-6e2e4546ca8f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:41.470674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88129e47-7da8-4f18-81fb-6e2e4546ca8f None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:41.472015 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 69/145] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:22:41.479467 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ec89e-91df-406a-addf-7a027218164a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:41.479756 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ec89e-91df-406a-addf-7a027218164a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:41.480034 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ec89e-91df-406a-addf-7a027218164a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:41.483590 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 77/146] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:41.504616 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e39ec89e-91df-406a-addf-7a027218164a None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:41.506330 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e39ec89e-91df-406a-addf-7a027218164a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:41.543974 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:41.598121 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848e3c5-f853-4b3a-a952-812ec1cc40d3 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:41.603720 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 70/147] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:41.653143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:41.707052 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848e3c5-f853-4b3a-a952-812ec1cc40d3 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:41.741862 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4b42a-d77e-49bf-bd67-5d0ad8ee4a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:41.742417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4b42a-d77e-49bf-bd67-5d0ad8ee4a91 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:41.742692 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4b42a-d77e-49bf-bd67-5d0ad8ee4a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:41.742960 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4b42a-d77e-49bf-bd67-5d0ad8ee4a91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:41.753239 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb4b42a-d77e-49bf-bd67-5d0ad8ee4a91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:41.758621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-2eb4b42a-d77e-49bf-bd67-5d0ad8ee4a91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:22:41.758621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:41.758621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:41.758621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:41.758621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:41.758621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:41.758621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:41.758621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:41.758621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:41.758621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:41.758621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:41.758621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:41.758621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:41.758621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:41.758621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:41.758621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:41.759442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:41.759442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:41.759442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:41.759442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:41.759442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:41.759442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:41.759442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:41.759442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:41.759442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:41.759442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:41.759442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:41.759442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:41.759442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:41.759442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:41.759442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:41.759442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:41.760326 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:41.760326 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:41.760326 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:41.760326 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:41.760326 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:41.760326 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:41.760326 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:41.760326 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:41.760326 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:41.760326 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:41.760326 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:22:41.760326 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:22:41.760326 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 78/148] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 20 13:22:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:22:41.767086 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ec89e-91df-406a-addf-7a027218164a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:41.770599 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4b42a-d77e-49bf-bd67-5d0ad8ee4a91 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:41.871961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ec89e-91df-406a-addf-7a027218164a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:41.876820 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4b42a-d77e-49bf-bd67-5d0ad8ee4a91 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:41.890224 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:41.907513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13fd1f-37f0-4f5a-bf3f-740351cbfcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:41.908127 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13fd1f-37f0-4f5a-bf3f-740351cbfcb2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:41.908415 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13fd1f-37f0-4f5a-bf3f-740351cbfcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:41.908725 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13fd1f-37f0-4f5a-bf3f-740351cbfcb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:41.909439 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de13fd1f-37f0-4f5a-bf3f-740351cbfcb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:41.909752 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de13fd1f-37f0-4f5a-bf3f-740351cbfcb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:41.911138 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de13fd1f-37f0-4f5a-bf3f-740351cbfcb2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:41.913190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbb8e8-2491-4dae-bba2-6204ebc58563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:41.913815 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbb8e8-2491-4dae-bba2-6204ebc58563 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:41.914091 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbb8e8-2491-4dae-bba2-6204ebc58563 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:41.914357 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbb8e8-2491-4dae-bba2-6204ebc58563 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:41.919775 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 71/149] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:22:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:41.920568 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13fd1f-37f0-4f5a-bf3f-740351cbfcb2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:41.926648 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbb8e8-2491-4dae-bba2-6204ebc58563 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:41.932438 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-57fbb8e8-2491-4dae-bba2-6204ebc58563 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 13:22:41.932438 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:41.932438 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:41.932438 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:41.932438 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:41.932438 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:41.932438 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:41.932438 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:41.932438 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:41.932438 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:41.932438 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:41.932438 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:41.932438 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:41.932438 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:41.932438 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:41.932438 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:41.933353 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:41.933353 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:41.933353 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:41.933353 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:41.933353 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:41.933353 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:41.933353 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:41.933353 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:41.933353 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:41.933353 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:41.933353 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:41.933353 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:41.933353 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:41.933353 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:41.933353 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:41.933353 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:41.934105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:41.934105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:41.934105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:41.934105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:41.934105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:41.934105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:41.934105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:41.934105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:41.934105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:41.934105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:41.934105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 13:22:41.934105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:22:41.934752 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 79/150] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 20 13:22:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:22:41.935104 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbb8e8-2491-4dae-bba2-6204ebc58563 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:41.950289 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:42.021375 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de13fd1f-37f0-4f5a-bf3f-740351cbfcb2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:42.032547 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbb8e8-2491-4dae-bba2-6204ebc58563 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:42.054597 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a80869-f66d-4f4b-b5e1-c99459d085a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:42.055223 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a80869-f66d-4f4b-b5e1-c99459d085a8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:42.055503 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a80869-f66d-4f4b-b5e1-c99459d085a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:42.055774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a80869-f66d-4f4b-b5e1-c99459d085a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:42.056915 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a80869-f66d-4f4b-b5e1-c99459d085a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:42.058510 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a80869-f66d-4f4b-b5e1-c99459d085a8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:42.060140 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 72/151] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:22:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:42.061152 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a80869-f66d-4f4b-b5e1-c99459d085a8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:42.071147 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ccb9a-7ace-4341-83de-cb7e769acb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:42.071858 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ccb9a-7ace-4341-83de-cb7e769acb3f None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:42.072205 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ccb9a-7ace-4341-83de-cb7e769acb3f None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:42.072542 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ccb9a-7ace-4341-83de-cb7e769acb3f None admin] PATH_INFO: `/v3/users/1138213b4cd745dda1aa1c81021ea0ce` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:42.087696 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ccb9a-7ace-4341-83de-cb7e769acb3f None admin] RBAC: Authorizing `identity:get_user(user_id=1138213b4cd745dda1aa1c81021ea0ce)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:42.089202 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ccb9a-7ace-4341-83de-cb7e769acb3f None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:42.090838 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 80/152] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:41 2022] GET /identity/v3/users/1138213b4cd745dda1aa1c81021ea0ce => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:42.091575 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ccb9a-7ace-4341-83de-cb7e769acb3f None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:42.119600 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:42.168800 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:42.175326 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a80869-f66d-4f4b-b5e1-c99459d085a8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:42.187002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ccb9a-7ace-4341-83de-cb7e769acb3f None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:42.208711 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae6dde-1d89-43f3-90d6-9804f2eb942b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:42.209445 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae6dde-1d89-43f3-90d6-9804f2eb942b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:42.209733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae6dde-1d89-43f3-90d6-9804f2eb942b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:42.210068 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae6dde-1d89-43f3-90d6-9804f2eb942b None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:42.210689 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae6dde-1d89-43f3-90d6-9804f2eb942b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:42.210997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae6dde-1d89-43f3-90d6-9804f2eb942b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:42.212373 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae6dde-1d89-43f3-90d6-9804f2eb942b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:42.219794 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bbe60-d0d6-451e-8c00-1bb495c9e3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:42.220392 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bbe60-d0d6-451e-8c00-1bb495c9e3ee None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:42.220669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bbe60-d0d6-451e-8c00-1bb495c9e3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:42.220971 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bbe60-d0d6-451e-8c00-1bb495c9e3ee None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:42.221676 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815bbe60-d0d6-451e-8c00-1bb495c9e3ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:42.221864 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 73/153] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:22:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:42.222657 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae6dde-1d89-43f3-90d6-9804f2eb942b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:42.223120 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815bbe60-d0d6-451e-8c00-1bb495c9e3ee None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:42.245976 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 81/154] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 20 13:22:42 2022] POST /identity/v3/users => generated 297 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:22:42.246797 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bbe60-d0d6-451e-8c00-1bb495c9e3ee None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:42.332998 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae6dde-1d89-43f3-90d6-9804f2eb942b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:42.358393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:42.375271 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48862e8b-ecc1-42f5-8109-ac5e80fc4455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:42.375925 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48862e8b-ecc1-42f5-8109-ac5e80fc4455 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:42.376209 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48862e8b-ecc1-42f5-8109-ac5e80fc4455 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:42.376487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48862e8b-ecc1-42f5-8109-ac5e80fc4455 None admin] PATH_INFO: `/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:42.376744 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bbe60-d0d6-451e-8c00-1bb495c9e3ee None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:42.377753 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48862e8b-ecc1-42f5-8109-ac5e80fc4455 None admin] RBAC: Authorizing `identity:get_project(project_id=1fd601dd45764a5ba6b4ac811eaf3eeb)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:42.379246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48862e8b-ecc1-42f5-8109-ac5e80fc4455 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:42.381320 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 74/155] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:22:42 2022] GET /identity/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:42.386977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48862e8b-ecc1-42f5-8109-ac5e80fc4455 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:42.417917 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:42.421341 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4d5a2-c0bd-46cf-ab65-a8db351d79cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:42.421998 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4d5a2-c0bd-46cf-ab65-a8db351d79cd None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:42.422283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4d5a2-c0bd-46cf-ab65-a8db351d79cd None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:42.422561 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4d5a2-c0bd-46cf-ab65-a8db351d79cd None admin] PATH_INFO: `/v3/users/1138213b4cd745dda1aa1c81021ea0ce` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:42.423720 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4d5a2-c0bd-46cf-ab65-a8db351d79cd None admin] RBAC: Authorizing `identity:get_user(user_id=1138213b4cd745dda1aa1c81021ea0ce)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:42.425221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4d5a2-c0bd-46cf-ab65-a8db351d79cd None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:42.426821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 82/156] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:42 2022] GET /identity/v3/users/1138213b4cd745dda1aa1c81021ea0ce => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:42.433035 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4d5a2-c0bd-46cf-ab65-a8db351d79cd None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:42.509910 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48862e8b-ecc1-42f5-8109-ac5e80fc4455 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:42.551971 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4d5a2-c0bd-46cf-ab65-a8db351d79cd None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:42.561600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaca3ca-1175-44ab-9b92-8621f4cd86e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:42.562275 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaca3ca-1175-44ab-9b92-8621f4cd86e7 None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:42.562667 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaca3ca-1175-44ab-9b92-8621f4cd86e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:42.562989 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaca3ca-1175-44ab-9b92-8621f4cd86e7 None admin] PATH_INFO: `/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb/users/1138213b4cd745dda1aa1c81021ea0ce/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:42.565401 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaca3ca-1175-44ab-9b92-8621f4cd86e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=1fd601dd45764a5ba6b4ac811eaf3eeb, user_id=1138213b4cd745dda1aa1c81021ea0ce, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:42.567186 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaca3ca-1175-44ab-9b92-8621f4cd86e7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:42.585031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 75/157] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:22:42 2022] PUT /identity/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb/users/1138213b4cd745dda1aa1c81021ea0ce/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:22:42.598819 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbed270-e6a9-4414-9458-9bdc21bee1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:42.600572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbed270-e6a9-4414-9458-9bdc21bee1c2 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:42.600892 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbed270-e6a9-4414-9458-9bdc21bee1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:42.601217 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbed270-e6a9-4414-9458-9bdc21bee1c2 None admin] PATH_INFO: `/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:42.602562 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbed270-e6a9-4414-9458-9bdc21bee1c2 None admin] RBAC: Authorizing `identity:get_project(project_id=1fd601dd45764a5ba6b4ac811eaf3eeb)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:42.604085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbed270-e6a9-4414-9458-9bdc21bee1c2 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:42.606330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 83/158] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:22:42 2022] GET /identity/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:22:42.612283 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaca3ca-1175-44ab-9b92-8621f4cd86e7 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:42.715723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:42.733506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaca3ca-1175-44ab-9b92-8621f4cd86e7 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:42.778594 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809843fb-f85c-465d-b37d-817164c08f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:42.779280 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809843fb-f85c-465d-b37d-817164c08f7c None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:42.779559 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809843fb-f85c-465d-b37d-817164c08f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:42.779832 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809843fb-f85c-465d-b37d-817164c08f7c None admin] PATH_INFO: `/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb/users/1138213b4cd745dda1aa1c81021ea0ce/roles/b0ccf031da9e4a7492c567b607ceb21f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:42.782016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809843fb-f85c-465d-b37d-817164c08f7c None admin] RBAC: Authorizing `identity:create_grant(project_id=1fd601dd45764a5ba6b4ac811eaf3eeb, user_id=1138213b4cd745dda1aa1c81021ea0ce, role_id=b0ccf031da9e4a7492c567b607ceb21f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:42.783474 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809843fb-f85c-465d-b37d-817164c08f7c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:42.799685 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 76/159] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:22:42 2022] PUT /identity/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb/users/1138213b4cd745dda1aa1c81021ea0ce/roles/b0ccf031da9e4a7492c567b607ceb21f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:22:44.609007 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929a9a7-4c58-4a29-98ac-24e9be06222f None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:44.609988 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929a9a7-4c58-4a29-98ac-24e9be06222f None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:44.610550 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929a9a7-4c58-4a29-98ac-24e9be06222f None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:44.612139 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 84/160] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:44.618079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20a12d-8d68-424b-a66a-d10af9a75ee5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:44.618502 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20a12d-8d68-424b-a66a-d10af9a75ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:44.618675 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c20a12d-8d68-424b-a66a-d10af9a75ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:44.621251 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508051e-c774-4e57-97b7-0f563114ca10 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:44.621544 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508051e-c774-4e57-97b7-0f563114ca10 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:44.621833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508051e-c774-4e57-97b7-0f563114ca10 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:44.623251 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 85/161] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:44.628089 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575dafb3-3aae-41ba-a246-f69945946cf5 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:44.628373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575dafb3-3aae-41ba-a246-f69945946cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:44.628648 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575dafb3-3aae-41ba-a246-f69945946cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:44.643323 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-3c20a12d-8d68-424b-a66a-d10af9a75ee5 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:44.644797 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3c20a12d-8d68-424b-a66a-d10af9a75ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:44.647715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-575dafb3-3aae-41ba-a246-f69945946cf5 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:44.648889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-575dafb3-3aae-41ba-a246-f69945946cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:44.675670 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:44.719736 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 86/162] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 13:22:44.720510 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 77/163] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:44.867131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf827e-62df-4e57-87e8-4bcd85aad875 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:44.867416 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf827e-62df-4e57-87e8-4bcd85aad875 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:44.867695 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf827e-62df-4e57-87e8-4bcd85aad875 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:44.869245 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 87/164] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:44.874436 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f46cd-e66c-4b59-ae6f-995ec2478a1b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:44.874727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f46cd-e66c-4b59-ae6f-995ec2478a1b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:44.875000 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f46cd-e66c-4b59-ae6f-995ec2478a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:44.894695 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c57f46cd-e66c-4b59-ae6f-995ec2478a1b None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:44.895916 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c57f46cd-e66c-4b59-ae6f-995ec2478a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:44.972674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 78/165] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 13:22:44.996193 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf827e-62df-4e57-87e8-4bcd85aad875 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:45.006859 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac2f0b-1e71-4c71-b1b2-f5654805e378 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:45.007351 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac2f0b-1e71-4c71-b1b2-f5654805e378 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:45.007462 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac2f0b-1e71-4c71-b1b2-f5654805e378 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:45.008925 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 79/166] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:22:45.019064 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa6b8c-5daa-4166-a179-036c245d72b3 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:45.019064 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa6b8c-5daa-4166-a179-036c245d72b3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:45.019064 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa6b8c-5daa-4166-a179-036c245d72b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:45.031276 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:45.040089 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-22fa6b8c-5daa-4166-a179-036c245d72b3 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:45.041930 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-22fa6b8c-5daa-4166-a179-036c245d72b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:45.106909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf827e-62df-4e57-87e8-4bcd85aad875 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:45.119645 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 80/167] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:45.126136 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa6b8c-5daa-4166-a179-036c245d72b3 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:45.143155 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d522e-c5ed-4efc-beac-487bd9629849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:45.144027 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d522e-c5ed-4efc-beac-487bd9629849 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:45.144471 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d522e-c5ed-4efc-beac-487bd9629849 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:45.144960 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d522e-c5ed-4efc-beac-487bd9629849 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:45.155416 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d522e-c5ed-4efc-beac-487bd9629849 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:45.160376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-ce0d522e-c5ed-4efc-beac-487bd9629849 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:22:45.160376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:45.160376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:45.160376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:45.160376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:45.160376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:45.160376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:45.160376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:45.160376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:45.160376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:45.160376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:45.160376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:45.160376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:45.160376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:45.160376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:45.160376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:45.161241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:45.161241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:45.161241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:45.161241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:45.161241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:45.161241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:45.161241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:45.161241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:45.161241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:45.161241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:45.161241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:45.161241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:45.161241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:45.161241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:45.161241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:45.161241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:45.162536 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:45.162536 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:45.162536 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:45.162536 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:45.162536 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:45.162536 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:45.162536 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:45.162536 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:45.162536 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:45.162536 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:45.162536 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:22:45.162536 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:22:45.164909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 88/168] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:22:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:22:45.173764 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d522e-c5ed-4efc-beac-487bd9629849 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:45.195172 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:45.260339 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa6b8c-5daa-4166-a179-036c245d72b3 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:45.299103 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc3422-e07c-43fb-8400-72ca7fcbb4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:45.299741 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc3422-e07c-43fb-8400-72ca7fcbb4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:45.299856 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d522e-c5ed-4efc-beac-487bd9629849 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:45.300088 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc3422-e07c-43fb-8400-72ca7fcbb4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:45.300290 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc3422-e07c-43fb-8400-72ca7fcbb4a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:45.301426 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc3422-e07c-43fb-8400-72ca7fcbb4a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:45.303491 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc3422-e07c-43fb-8400-72ca7fcbb4a4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:45.305335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 81/169] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:22:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:45.305854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc3422-e07c-43fb-8400-72ca7fcbb4a4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:45.310412 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:45.335139 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7b586-a6e3-4a05-8358-8ff5596cb5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:45.335765 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7b586-a6e3-4a05-8358-8ff5596cb5f0 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:45.336042 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7b586-a6e3-4a05-8358-8ff5596cb5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:45.336307 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b7b586-a6e3-4a05-8358-8ff5596cb5f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:45.337002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7b586-a6e3-4a05-8358-8ff5596cb5f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:45.337303 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7b586-a6e3-4a05-8358-8ff5596cb5f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:45.338683 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b7b586-a6e3-4a05-8358-8ff5596cb5f0 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:45.346799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 89/170] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 20 13:22:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:45.346799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7b586-a6e3-4a05-8358-8ff5596cb5f0 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:45.419093 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc3422-e07c-43fb-8400-72ca7fcbb4a4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:45.426080 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:45.450760 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b7b586-a6e3-4a05-8358-8ff5596cb5f0 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:45.452579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ccf15-3c6e-4420-aeae-dd91ac6a5e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:45.453236 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ccf15-3c6e-4420-aeae-dd91ac6a5e72 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:45.453510 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ccf15-3c6e-4420-aeae-dd91ac6a5e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:45.453778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ccf15-3c6e-4420-aeae-dd91ac6a5e72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:45.464138 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ccf15-3c6e-4420-aeae-dd91ac6a5e72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:45.469386 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-c22ccf15-3c6e-4420-aeae-dd91ac6a5e72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:22:45.469386 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:45.469386 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:45.469386 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:45.469386 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:45.469386 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:45.469386 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:45.469386 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:45.469386 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:45.469386 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:45.469386 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:45.469386 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:45.469386 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:45.469386 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:45.469386 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:45.469386 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:45.470228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:45.470228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:45.470228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:45.470228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:45.470228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:45.470228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:45.470228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:45.470228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:45.470228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:45.470228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:45.470228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:45.470228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:45.470228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:45.470228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:45.470228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:45.470228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:45.470984 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:45.470984 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:45.470984 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:45.470984 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:45.470984 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:45.470984 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:45.470984 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:45.470984 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:45.470984 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:45.470984 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:45.470984 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:22:45.470984 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:22:45.470984 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 82/171] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 20 13:22:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:22:45.471619 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ccf15-3c6e-4420-aeae-dd91ac6a5e72 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:45.485985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394638ca-5d27-4a04-a56b-314f232a9671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:45.486591 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394638ca-5d27-4a04-a56b-314f232a9671 None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:45.486870 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394638ca-5d27-4a04-a56b-314f232a9671 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:45.487142 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394638ca-5d27-4a04-a56b-314f232a9671 None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:45.487792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394638ca-5d27-4a04-a56b-314f232a9671 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:45.489279 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394638ca-5d27-4a04-a56b-314f232a9671 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:45.510567 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 90/172] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 20 13:22:45 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:22:45.511406 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394638ca-5d27-4a04-a56b-314f232a9671 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:45.547280 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:45.591251 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ccf15-3c6e-4420-aeae-dd91ac6a5e72 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:45.629060 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394638ca-5d27-4a04-a56b-314f232a9671 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:45.631271 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53e73f-769a-419d-bf00-985b9ee7bd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:45.632624 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53e73f-769a-419d-bf00-985b9ee7bd94 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:45.632624 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53e73f-769a-419d-bf00-985b9ee7bd94 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:45.632624 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53e73f-769a-419d-bf00-985b9ee7bd94 None admin] PATH_INFO: `/v3/users/102a304865f94cb1971b73ffeccd78ae` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:45.648389 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c53e73f-769a-419d-bf00-985b9ee7bd94 None admin] RBAC: Authorizing `identity:get_user(user_id=102a304865f94cb1971b73ffeccd78ae)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:45.650024 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c53e73f-769a-419d-bf00-985b9ee7bd94 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:45.651911 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 83/173] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:45 2022] GET /identity/v3/users/102a304865f94cb1971b73ffeccd78ae => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:45.654244 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53e73f-769a-419d-bf00-985b9ee7bd94 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:45.661679 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:45.669349 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9877c-9203-40ed-adeb-4a459b96da97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:45.669855 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9877c-9203-40ed-adeb-4a459b96da97 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:45.670398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9877c-9203-40ed-adeb-4a459b96da97 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:45.670671 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9877c-9203-40ed-adeb-4a459b96da97 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:45.682952 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b9877c-9203-40ed-adeb-4a459b96da97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:45.689068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-c1b9877c-9203-40ed-adeb-4a459b96da97 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 13:22:45.689068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:45.689068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:45.689068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:45.689068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:45.689068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:45.689068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:45.689068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:45.689068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:45.689068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:45.689068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:45.689068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:45.689068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:45.689068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:45.689068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:45.689068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:45.689940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:45.689940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:45.689940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:45.689940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:45.689940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:45.689940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:45.689940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:45.689940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:45.689940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:45.689940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:45.689940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:45.689940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:45.689940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:45.689940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:45.689940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:45.689940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:45.690724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:45.690724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:45.690724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:45.690724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:45.690724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:45.690724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:45.690724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:45.690724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:45.690724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:45.690724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:45.690724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 13:22:45.690724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:22:45.690724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 91/174] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 20 13:22:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:22:45.690724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9877c-9203-40ed-adeb-4a459b96da97 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:45.784120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c53e73f-769a-419d-bf00-985b9ee7bd94 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:45.806531 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9877c-9203-40ed-adeb-4a459b96da97 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:45.814547 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:45.832391 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029a8e4-9ead-4f92-9cb0-b07e79af1fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:45.833031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029a8e4-9ead-4f92-9cb0-b07e79af1fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:45.833311 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029a8e4-9ead-4f92-9cb0-b07e79af1fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:45.833907 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029a8e4-9ead-4f92-9cb0-b07e79af1fd5 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:45.834657 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8029a8e4-9ead-4f92-9cb0-b07e79af1fd5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:45.834963 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8029a8e4-9ead-4f92-9cb0-b07e79af1fd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:45.836354 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8029a8e4-9ead-4f92-9cb0-b07e79af1fd5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:45.846072 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 84/175] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:22:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:45.850118 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029a8e4-9ead-4f92-9cb0-b07e79af1fd5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:45.861210 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585f5a9-1250-46f9-a86d-de1294bb6497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:45.861851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585f5a9-1250-46f9-a86d-de1294bb6497 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:45.862127 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585f5a9-1250-46f9-a86d-de1294bb6497 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:45.862902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585f5a9-1250-46f9-a86d-de1294bb6497 None admin] PATH_INFO: `/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:45.864341 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7585f5a9-1250-46f9-a86d-de1294bb6497 None admin] RBAC: Authorizing `identity:get_project(project_id=1fd601dd45764a5ba6b4ac811eaf3eeb)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:45.866077 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7585f5a9-1250-46f9-a86d-de1294bb6497 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:45.867800 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 92/176] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:22:45 2022] GET /identity/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:45.869807 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585f5a9-1250-46f9-a86d-de1294bb6497 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:45.939717 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:45.975465 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029a8e4-9ead-4f92-9cb0-b07e79af1fd5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:45.987201 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7585f5a9-1250-46f9-a86d-de1294bb6497 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:46.016843 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339ac30-ff46-4669-959c-d710d8e1293f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:46.017586 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339ac30-ff46-4669-959c-d710d8e1293f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:46.017869 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339ac30-ff46-4669-959c-d710d8e1293f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:46.018260 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339ac30-ff46-4669-959c-d710d8e1293f None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:46.019055 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9339ac30-ff46-4669-959c-d710d8e1293f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:46.019430 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9339ac30-ff46-4669-959c-d710d8e1293f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:46.020890 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9339ac30-ff46-4669-959c-d710d8e1293f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:46.023972 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379fdff9-1a17-44de-90a4-5f5fcc4e290e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:46.024693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fdff9-1a17-44de-90a4-5f5fcc4e290e None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:46.025015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fdff9-1a17-44de-90a4-5f5fcc4e290e None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:46.025297 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fdff9-1a17-44de-90a4-5f5fcc4e290e None admin] PATH_INFO: `/v3/users/1138213b4cd745dda1aa1c81021ea0ce` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:46.026449 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379fdff9-1a17-44de-90a4-5f5fcc4e290e None admin] RBAC: Authorizing `identity:get_user(user_id=1138213b4cd745dda1aa1c81021ea0ce)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:46.028083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379fdff9-1a17-44de-90a4-5f5fcc4e290e None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:46.030048 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 93/177] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:45 2022] GET /identity/v3/users/1138213b4cd745dda1aa1c81021ea0ce => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:46.030590 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379fdff9-1a17-44de-90a4-5f5fcc4e290e None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:46.030766 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 85/178] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:22:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:46.036100 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339ac30-ff46-4669-959c-d710d8e1293f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:46.095375 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:46.154721 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339ac30-ff46-4669-959c-d710d8e1293f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:46.167853 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379fdff9-1a17-44de-90a4-5f5fcc4e290e None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:46.191107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c1c6ba-8674-4edc-b6fd-1b1e2ac8357b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:46.191675 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1c6ba-8674-4edc-b6fd-1b1e2ac8357b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:46.191957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1c6ba-8674-4edc-b6fd-1b1e2ac8357b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:46.192322 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1c6ba-8674-4edc-b6fd-1b1e2ac8357b None admin] PATH_INFO: `/v3/users/1138213b4cd745dda1aa1c81021ea0ce` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:46.193807 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c1c6ba-8674-4edc-b6fd-1b1e2ac8357b None admin] RBAC: Authorizing `identity:get_user(user_id=1138213b4cd745dda1aa1c81021ea0ce)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:46.195193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c1c6ba-8674-4edc-b6fd-1b1e2ac8357b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:46.197183 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 86/179] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:46 2022] GET /identity/v3/users/1138213b4cd745dda1aa1c81021ea0ce => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:22:46.197699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c1c6ba-8674-4edc-b6fd-1b1e2ac8357b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:46.199065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:46.210557 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e71a36-8d24-4cc1-8ee5-b1c460f5bcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:46.211399 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e71a36-8d24-4cc1-8ee5-b1c460f5bcca None admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:46.211782 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e71a36-8d24-4cc1-8ee5-b1c460f5bcca None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:46.212153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e71a36-8d24-4cc1-8ee5-b1c460f5bcca None admin] PATH_INFO: `/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb/users/102a304865f94cb1971b73ffeccd78ae/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:46.214358 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e71a36-8d24-4cc1-8ee5-b1c460f5bcca None admin] RBAC: Authorizing `identity:create_grant(project_id=1fd601dd45764a5ba6b4ac811eaf3eeb, user_id=102a304865f94cb1971b73ffeccd78ae, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:46.216067 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e71a36-8d24-4cc1-8ee5-b1c460f5bcca None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:46.240927 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 94/180] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:22:46 2022] PUT /identity/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb/users/102a304865f94cb1971b73ffeccd78ae/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:22:46.240927 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e71a36-8d24-4cc1-8ee5-b1c460f5bcca None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:46.398339 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c1c6ba-8674-4edc-b6fd-1b1e2ac8357b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:46.407955 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e71a36-8d24-4cc1-8ee5-b1c460f5bcca None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:46.426366 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:46.463641 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4175a7-5405-40b3-bd8a-c3909bc95fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:46.465375 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01e8fd-8739-4ea2-bf62-db7f6a1196cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:46.466172 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4175a7-5405-40b3-bd8a-c3909bc95fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:46.466662 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4175a7-5405-40b3-bd8a-c3909bc95fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:46.467523 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01e8fd-8739-4ea2-bf62-db7f6a1196cc None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:46.468005 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4175a7-5405-40b3-bd8a-c3909bc95fb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:46.469150 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01e8fd-8739-4ea2-bf62-db7f6a1196cc None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:46.469653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01e8fd-8739-4ea2-bf62-db7f6a1196cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:46.557943 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4175a7-5405-40b3-bd8a-c3909bc95fb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:46.558972 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:46.560145 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4175a7-5405-40b3-bd8a-c3909bc95fb4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:46.578419 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01e8fd-8739-4ea2-bf62-db7f6a1196cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:46.579914 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f01e8fd-8739-4ea2-bf62-db7f6a1196cc None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:46.676468 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 87/181] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:22:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 13:22:46.682782 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4175a7-5405-40b3-bd8a-c3909bc95fb4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:46.698763 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:46.787580 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4175a7-5405-40b3-bd8a-c3909bc95fb4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:46.823399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e8323-f3a0-49eb-afe7-04b6db9ba36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:46.824439 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e8323-f3a0-49eb-afe7-04b6db9ba36b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:46.824960 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e8323-f3a0-49eb-afe7-04b6db9ba36b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:46.825682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e8323-f3a0-49eb-afe7-04b6db9ba36b None admin] PATH_INFO: `/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:46.827071 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301e8323-f3a0-49eb-afe7-04b6db9ba36b None admin] RBAC: Authorizing `identity:get_project(project_id=1fd601dd45764a5ba6b4ac811eaf3eeb)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:46.828741 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301e8323-f3a0-49eb-afe7-04b6db9ba36b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:46.830615 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 88/182] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:22:46 2022] GET /identity/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:22:46.832691 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 95/183] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:22:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 13:22:46.842415 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e8323-f3a0-49eb-afe7-04b6db9ba36b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:46.842559 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01e8fd-8739-4ea2-bf62-db7f6a1196cc None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:46.995784 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:47.002725 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:47.025669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01e8fd-8739-4ea2-bf62-db7f6a1196cc None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:47.039700 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e8323-f3a0-49eb-afe7-04b6db9ba36b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:47.078269 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780513b-5284-432f-b376-48338888ade6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:47.078876 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780513b-5284-432f-b376-48338888ade6 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:47.079157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780513b-5284-432f-b376-48338888ade6 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:47.079426 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780513b-5284-432f-b376-48338888ade6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:47.080154 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3780513b-5284-432f-b376-48338888ade6 None admin] RBAC: Adding query filter params (role.id=2bcf35e1bc5f4634b0aa6411fad2f43f, scope.project.id=1fd601dd45764a5ba6b4ac811eaf3eeb, user.id=1138213b4cd745dda1aa1c81021ea0ce) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:47.080458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3780513b-5284-432f-b376-48338888ade6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:47.081950 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3780513b-5284-432f-b376-48338888ade6 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:47.088446 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 96/184] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:22:46 2022] GET /identity/v3/role_assignments?user.id=1138213b4cd745dda1aa1c81021ea0ce&scope.project.id=1fd601dd45764a5ba6b4ac811eaf3eeb&role.id=2bcf35e1bc5f4634b0aa6411fad2f43f => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:47.109645 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec65cb08-5eed-4ecc-885b-0df13965c374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:47.110312 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65cb08-5eed-4ecc-885b-0df13965c374 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:47.110599 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65cb08-5eed-4ecc-885b-0df13965c374 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:47.110875 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65cb08-5eed-4ecc-885b-0df13965c374 None admin] PATH_INFO: `/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:47.112152 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec65cb08-5eed-4ecc-885b-0df13965c374 None admin] RBAC: Authorizing `identity:get_project(project_id=1fd601dd45764a5ba6b4ac811eaf3eeb)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:47.113661 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec65cb08-5eed-4ecc-885b-0df13965c374 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:47.116063 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 89/185] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:22:46 2022] GET /identity/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:47.122328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780513b-5284-432f-b376-48338888ade6 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:47.296775 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780513b-5284-432f-b376-48338888ade6 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:47.332195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6ea19-4c74-43d6-be2d-ec4d2753d165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:47.332914 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6ea19-4c74-43d6-be2d-ec4d2753d165 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:47.333264 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6ea19-4c74-43d6-be2d-ec4d2753d165 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:47.333606 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6ea19-4c74-43d6-be2d-ec4d2753d165 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:47.334991 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb6ea19-4c74-43d6-be2d-ec4d2753d165 None admin] RBAC: Adding query filter params (role.id=b0ccf031da9e4a7492c567b607ceb21f, scope.project.id=1fd601dd45764a5ba6b4ac811eaf3eeb, user.id=1138213b4cd745dda1aa1c81021ea0ce) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:47.335373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb6ea19-4c74-43d6-be2d-ec4d2753d165 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:47.336903 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb6ea19-4c74-43d6-be2d-ec4d2753d165 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:47.342938 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 97/186] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:22:47 2022] GET /identity/v3/role_assignments?user.id=1138213b4cd745dda1aa1c81021ea0ce&scope.project.id=1fd601dd45764a5ba6b4ac811eaf3eeb&role.id=b0ccf031da9e4a7492c567b607ceb21f => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:47.898493 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a6a80-8e4a-4599-99cc-8c86325fcbde None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:47.899003 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a6a80-8e4a-4599-99cc-8c86325fcbde None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:47.899076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a6a80-8e4a-4599-99cc-8c86325fcbde None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:47.900513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 90/187] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:47.905400 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdcb5c-1e7f-4460-9bca-e2994d608af6 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:47.905779 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdcb5c-1e7f-4460-9bca-e2994d608af6 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:47.905948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdcb5c-1e7f-4460-9bca-e2994d608af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:47.924402 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-c2cdcb5c-1e7f-4460-9bca-e2994d608af6 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:47.925623 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-c2cdcb5c-1e7f-4460-9bca-e2994d608af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:47.937869 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:48.000764 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 98/188] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:48.048111 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be6909-1640-4e8f-9d65-ec77a3fc43b6 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:48.048425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be6909-1640-4e8f-9d65-ec77a3fc43b6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:48.048667 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be6909-1640-4e8f-9d65-ec77a3fc43b6 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:48.050179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 91/189] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:48.055419 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a6fe0-ba30-4d6d-b3e0-0691cb15c33e None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:48.055713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a6fe0-ba30-4d6d-b3e0-0691cb15c33e None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:48.055995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a6fe0-ba30-4d6d-b3e0-0691cb15c33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:48.074729 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-748a6fe0-ba30-4d6d-b3e0-0691cb15c33e None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:48.075893 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-748a6fe0-ba30-4d6d-b3e0-0691cb15c33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:48.144200 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 99/190] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:48.274855 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be6909-1640-4e8f-9d65-ec77a3fc43b6 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:48.375152 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be6909-1640-4e8f-9d65-ec77a3fc43b6 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:48.408646 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e44907-2cd1-4f59-94c2-bca77f81e39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:48.409462 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e44907-2cd1-4f59-94c2-bca77f81e39b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:48.409837 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e44907-2cd1-4f59-94c2-bca77f81e39b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:48.410106 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e44907-2cd1-4f59-94c2-bca77f81e39b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:48.417967 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a6fe0-ba30-4d6d-b3e0-0691cb15c33e None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:48.420822 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e44907-2cd1-4f59-94c2-bca77f81e39b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:48.425962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d4e44907-2cd1-4f59-94c2-bca77f81e39b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 13:22:48.425962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:48.425962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:48.425962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:48.425962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:48.425962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:48.425962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:48.425962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:48.425962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:48.425962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:48.425962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:48.425962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:48.425962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:48.425962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:48.425962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:48.425962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:48.426996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:48.426996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:48.426996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:48.426996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:48.426996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:48.426996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:48.426996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:48.426996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:48.426996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:48.426996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:48.426996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:48.426996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:48.426996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:48.426996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:48.426996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:48.426996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:48.428457 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:48.428457 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:48.428457 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:48.428457 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:48.428457 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:48.428457 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:48.428457 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:48.428457 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:48.428457 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:48.428457 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:48.428457 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 13:22:48.428457 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:22:48.428457 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 92/191] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:22:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:22:48.433287 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e44907-2cd1-4f59-94c2-bca77f81e39b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:48.447251 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:48.498280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:48.531389 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a6fe0-ba30-4d6d-b3e0-0691cb15c33e None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:48.553921 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e44907-2cd1-4f59-94c2-bca77f81e39b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:48.571672 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674ba7d-9b1e-40a4-a053-cb0a421a6245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:48.572157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674ba7d-9b1e-40a4-a053-cb0a421a6245 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:48.572434 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674ba7d-9b1e-40a4-a053-cb0a421a6245 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:48.572704 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674ba7d-9b1e-40a4-a053-cb0a421a6245 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:48.573878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674ba7d-9b1e-40a4-a053-cb0a421a6245 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:48.575329 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674ba7d-9b1e-40a4-a053-cb0a421a6245 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:48.577064 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 100/192] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:22:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:22:48.582254 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674ba7d-9b1e-40a4-a053-cb0a421a6245 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:48.589854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c250fef4-271d-4776-a147-449bacb545df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:48.590465 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250fef4-271d-4776-a147-449bacb545df None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:48.590747 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250fef4-271d-4776-a147-449bacb545df None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:48.591026 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250fef4-271d-4776-a147-449bacb545df None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:48.591698 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c250fef4-271d-4776-a147-449bacb545df None admin] RBAC: Adding query filter params (name=reader) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:48.592008 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c250fef4-271d-4776-a147-449bacb545df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:48.593417 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c250fef4-271d-4776-a147-449bacb545df None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:48.602259 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 93/193] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 20 13:22:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:48.607559 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c250fef4-271d-4776-a147-449bacb545df None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:48.682028 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:48.689448 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674ba7d-9b1e-40a4-a053-cb0a421a6245 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:48.714692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c250fef4-271d-4776-a147-449bacb545df None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:48.722572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e5302d-f00b-4d1c-8659-ca0c4e4e6434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:48.723204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5302d-f00b-4d1c-8659-ca0c4e4e6434 None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:48.723480 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5302d-f00b-4d1c-8659-ca0c4e4e6434 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:48.723747 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5302d-f00b-4d1c-8659-ca0c4e4e6434 None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:48.724987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e5302d-f00b-4d1c-8659-ca0c4e4e6434 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:48.726703 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e5302d-f00b-4d1c-8659-ca0c4e4e6434 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:48.744938 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:48.756352 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e665cb-0da1-4389-bcc9-ffcd19061d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:48.757052 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e665cb-0da1-4389-bcc9-ffcd19061d61 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:48.757336 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e665cb-0da1-4389-bcc9-ffcd19061d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:48.757611 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e665cb-0da1-4389-bcc9-ffcd19061d61 None admin] PATH_INFO: `/v3/users/8da0b7fb45834462af76a71e2aec7eef` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:48.759915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 101/194] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 20 13:22:48 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:22:48.773785 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e665cb-0da1-4389-bcc9-ffcd19061d61 None admin] RBAC: Authorizing `identity:get_user(user_id=8da0b7fb45834462af76a71e2aec7eef)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:48.775592 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e665cb-0da1-4389-bcc9-ffcd19061d61 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:48.777596 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 94/195] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:48 2022] GET /identity/v3/users/8da0b7fb45834462af76a71e2aec7eef => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:48.783294 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e5302d-f00b-4d1c-8659-ca0c4e4e6434 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:48.847712 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a704052-20fe-43b6-8a2e-06c3da9f68f8 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:48.848056 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a704052-20fe-43b6-8a2e-06c3da9f68f8 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:48.848302 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a704052-20fe-43b6-8a2e-06c3da9f68f8 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:48.850489 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 95/196] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:22:48.855927 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b125a6d-a589-4e01-b243-70c136dc1eb2 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:48.856212 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b125a6d-a589-4e01-b243-70c136dc1eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:48.856494 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b125a6d-a589-4e01-b243-70c136dc1eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:48.875891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-3b125a6d-a589-4e01-b243-70c136dc1eb2 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:48.877142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3b125a6d-a589-4e01-b243-70c136dc1eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:48.898440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e5302d-f00b-4d1c-8659-ca0c4e4e6434 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:48.934103 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f4838-8318-412c-b063-f93445061345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:48.934902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f4838-8318-412c-b063-f93445061345 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:48.935010 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f4838-8318-412c-b063-f93445061345 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:48.935285 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f4838-8318-412c-b063-f93445061345 None admin] PATH_INFO: `/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:48.936439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f4838-8318-412c-b063-f93445061345 None admin] RBAC: Authorizing `identity:get_project(project_id=1fd601dd45764a5ba6b4ac811eaf3eeb)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:48.937937 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f4838-8318-412c-b063-f93445061345 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:48.939578 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 102/197] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:22:48 2022] GET /identity/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:48.946218 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f4838-8318-412c-b063-f93445061345 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:48.955909 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 96/198] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 13:22:49.022777 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:49.063091 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f4838-8318-412c-b063-f93445061345 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:49.101018 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5e52d-6624-42c2-bbdc-6cfc5b1f8db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:49.101610 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5e52d-6624-42c2-bbdc-6cfc5b1f8db5 None admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:49.101892 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5e52d-6624-42c2-bbdc-6cfc5b1f8db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:49.102164 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5e52d-6624-42c2-bbdc-6cfc5b1f8db5 None admin] PATH_INFO: `/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb/users/8da0b7fb45834462af76a71e2aec7eef/roles/83ca567af06444b3b95843cb52d61698` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:49.104069 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5e52d-6624-42c2-bbdc-6cfc5b1f8db5 None admin] RBAC: Authorizing `identity:create_grant(project_id=1fd601dd45764a5ba6b4ac811eaf3eeb, user_id=8da0b7fb45834462af76a71e2aec7eef, role_id=83ca567af06444b3b95843cb52d61698)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:49.105567 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5e52d-6624-42c2-bbdc-6cfc5b1f8db5 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:49.122527 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 103/199] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:22:48 2022] PUT /identity/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb/users/8da0b7fb45834462af76a71e2aec7eef/roles/83ca567af06444b3b95843cb52d61698 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:22:49.229206 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b125a6d-a589-4e01-b243-70c136dc1eb2 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:49.302172 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:49.346283 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b125a6d-a589-4e01-b243-70c136dc1eb2 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:49.383421 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00e74e-744a-4737-8452-11d99b88acab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:49.384064 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00e74e-744a-4737-8452-11d99b88acab None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:49.384348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00e74e-744a-4737-8452-11d99b88acab None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:49.384625 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00e74e-744a-4737-8452-11d99b88acab None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:49.395688 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00e74e-744a-4737-8452-11d99b88acab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:49.401273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-5b00e74e-744a-4737-8452-11d99b88acab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:22:49.401273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:49.401273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:49.401273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:49.401273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:49.401273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:49.401273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:49.401273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:49.401273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:49.401273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:49.401273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:49.401273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:49.401273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:49.401273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:49.401273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:49.401273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:49.402128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:49.402128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:49.402128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:49.402128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:49.402128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:49.402128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:49.402128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:49.402128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:49.402128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:49.402128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:49.402128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:49.402128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:49.402128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:49.402128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:49.402128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:49.402128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:49.403402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:49.403402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:49.403402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:49.403402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:49.403402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:49.403402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:49.403402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:49.403402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:49.403402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:49.403402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:49.403402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:22:49.403402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:22:49.403402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 97/200] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:22:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:22:49.408584 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5e52d-6624-42c2-bbdc-6cfc5b1f8db5 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:49.520799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5e52d-6624-42c2-bbdc-6cfc5b1f8db5 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:49.547477 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:49.558391 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929c6f8-a658-47f3-a6f7-0bedf4cb3a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:49.559008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929c6f8-a658-47f3-a6f7-0bedf4cb3a50 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:49.559286 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929c6f8-a658-47f3-a6f7-0bedf4cb3a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:49.559556 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929c6f8-a658-47f3-a6f7-0bedf4cb3a50 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:49.560219 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929c6f8-a658-47f3-a6f7-0bedf4cb3a50 None admin] RBAC: Adding query filter params (name=member) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:49.560518 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929c6f8-a658-47f3-a6f7-0bedf4cb3a50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:49.561947 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929c6f8-a658-47f3-a6f7-0bedf4cb3a50 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:49.571250 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 104/201] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 20 13:22:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:49.577057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00e74e-744a-4737-8452-11d99b88acab None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:49.684881 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00e74e-744a-4737-8452-11d99b88acab None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:49.706446 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:49.721903 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e83abf-9f02-41d6-b5a3-4bd5797ca859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:49.722870 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e83abf-9f02-41d6-b5a3-4bd5797ca859 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:49.723342 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e83abf-9f02-41d6-b5a3-4bd5797ca859 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:49.723801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e83abf-9f02-41d6-b5a3-4bd5797ca859 None admin] PATH_INFO: `/v3/users/102a304865f94cb1971b73ffeccd78ae` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:49.725803 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e83abf-9f02-41d6-b5a3-4bd5797ca859 None admin] RBAC: Authorizing `identity:get_user(user_id=102a304865f94cb1971b73ffeccd78ae)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:49.727479 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e83abf-9f02-41d6-b5a3-4bd5797ca859 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:49.729339 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 98/202] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:49 2022] GET /identity/v3/users/102a304865f94cb1971b73ffeccd78ae => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:49.737403 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929c6f8-a658-47f3-a6f7-0bedf4cb3a50 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:49.841316 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929c6f8-a658-47f3-a6f7-0bedf4cb3a50 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:49.877500 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a51fb-5806-4a6a-ac85-be7f51b79557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:49.878123 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a51fb-5806-4a6a-ac85-be7f51b79557 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:49.878403 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a51fb-5806-4a6a-ac85-be7f51b79557 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:49.878675 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a51fb-5806-4a6a-ac85-be7f51b79557 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:49.937344 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:49.949826 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a51fb-5806-4a6a-ac85-be7f51b79557 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:49.951333 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a51fb-5806-4a6a-ac85-be7f51b79557 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:50.055398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 105/203] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:22:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 13:22:50.056795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e83abf-9f02-41d6-b5a3-4bd5797ca859 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:50.168502 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e83abf-9f02-41d6-b5a3-4bd5797ca859 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:50.203891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458943e1-1f8c-4ab1-8f47-858f35338fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:50.204499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458943e1-1f8c-4ab1-8f47-858f35338fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:50.204778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458943e1-1f8c-4ab1-8f47-858f35338fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:50.205076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458943e1-1f8c-4ab1-8f47-858f35338fd4 None admin] PATH_INFO: `/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:50.206293 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458943e1-1f8c-4ab1-8f47-858f35338fd4 None admin] RBAC: Authorizing `identity:get_project(project_id=1fd601dd45764a5ba6b4ac811eaf3eeb)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:50.207726 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458943e1-1f8c-4ab1-8f47-858f35338fd4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:50.209597 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 99/204] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:22:50 2022] GET /identity/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:50.215531 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a51fb-5806-4a6a-ac85-be7f51b79557 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:50.308107 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:50.325706 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a51fb-5806-4a6a-ac85-be7f51b79557 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:50.361864 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa362f-3121-475b-9701-3141ffb8bced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:50.362780 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa362f-3121-475b-9701-3141ffb8bced None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:50.363279 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa362f-3121-475b-9701-3141ffb8bced None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:50.363772 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa362f-3121-475b-9701-3141ffb8bced None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:50.364685 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa362f-3121-475b-9701-3141ffb8bced None admin] RBAC: Adding query filter params (role.id=5c47f5b95d6f47a78adf8f8e91562e4b, scope.project.id=1fd601dd45764a5ba6b4ac811eaf3eeb, user.id=102a304865f94cb1971b73ffeccd78ae) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:50.365465 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa362f-3121-475b-9701-3141ffb8bced None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:50.367094 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa362f-3121-475b-9701-3141ffb8bced None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:50.373607 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 106/205] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:22:50 2022] GET /identity/v3/role_assignments?user.id=102a304865f94cb1971b73ffeccd78ae&scope.project.id=1fd601dd45764a5ba6b4ac811eaf3eeb&role.id=5c47f5b95d6f47a78adf8f8e91562e4b => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:51.140874 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee62555-8f57-472c-bfdc-69066de393e5 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:51.141385 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee62555-8f57-472c-bfdc-69066de393e5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:51.141456 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee62555-8f57-472c-bfdc-69066de393e5 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:51.142913 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 100/206] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:51.148309 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a234fa-de75-4f79-bae6-c2e9254cae27 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:51.148688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a234fa-de75-4f79-bae6-c2e9254cae27 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:51.148878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a234fa-de75-4f79-bae6-c2e9254cae27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:51.167383 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-a3a234fa-de75-4f79-bae6-c2e9254cae27 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:51.168565 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-a3a234fa-de75-4f79-bae6-c2e9254cae27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:51.192793 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a507bc-ffd1-4798-812f-340abbb5ed72 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:51.193785 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a507bc-ffd1-4798-812f-340abbb5ed72 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:51.194075 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a507bc-ffd1-4798-812f-340abbb5ed72 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:51.195546 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 101/207] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:22:51.200509 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f436169-fff9-4c07-951d-fd1dc0dd16ca None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:51.200802 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f436169-fff9-4c07-951d-fd1dc0dd16ca None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:51.201120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f436169-fff9-4c07-951d-fd1dc0dd16ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:51.219320 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5f436169-fff9-4c07-951d-fd1dc0dd16ca None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:51.220490 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5f436169-fff9-4c07-951d-fd1dc0dd16ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:51.238930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 107/208] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:51.273799 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:51.294742 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 102/209] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 13:22:51.395415 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c96d6-2eb5-45b2-8a11-fcf21d567f3a None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:51.395710 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c96d6-2eb5-45b2-8a11-fcf21d567f3a None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:51.395881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c96d6-2eb5-45b2-8a11-fcf21d567f3a None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:51.397812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 108/210] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:51.403001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f007c2-f10d-46a0-9650-656ee26ff6fb None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:51.403297 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f007c2-f10d-46a0-9650-656ee26ff6fb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:51.403574 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f007c2-f10d-46a0-9650-656ee26ff6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:51.421738 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-46f007c2-f10d-46a0-9650-656ee26ff6fb None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:51.422922 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-46f007c2-f10d-46a0-9650-656ee26ff6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:51.490427 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 103/211] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:51.508462 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c96d6-2eb5-45b2-8a11-fcf21d567f3a None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:51.572479 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f007c2-f10d-46a0-9650-656ee26ff6fb None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:51.591193 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:51.611785 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c96d6-2eb5-45b2-8a11-fcf21d567f3a None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:51.648883 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead93d7-827a-4dcd-b6b4-2181e02f67d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:51.649447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead93d7-827a-4dcd-b6b4-2181e02f67d8 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:51.649727 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead93d7-827a-4dcd-b6b4-2181e02f67d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:51.650120 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead93d7-827a-4dcd-b6b4-2181e02f67d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:51.660637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead93d7-827a-4dcd-b6b4-2181e02f67d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:51.665915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-fead93d7-827a-4dcd-b6b4-2181e02f67d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:22:51.665915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:51.665915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:51.665915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:51.665915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:51.665915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:51.665915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:51.665915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:51.665915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:51.665915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:51.665915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:51.665915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:51.665915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:51.665915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:51.665915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:51.665915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:51.666749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:51.666749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:51.666749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:51.666749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:51.666749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:51.666749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:51.666749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:51.666749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:51.666749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:51.666749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:51.666749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:51.666749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:51.666749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:51.666749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:51.666749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:51.666749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:51.667519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:51.667519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:51.667519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:51.667519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:51.667519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:51.667519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:51.667519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:51.667519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:51.667519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:51.667519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:51.667519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:22:51.667519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:22:51.667519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 109/212] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:22:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:22:51.672933 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead93d7-827a-4dcd-b6b4-2181e02f67d8 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:51.675134 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f007c2-f10d-46a0-9650-656ee26ff6fb None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:51.694100 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:51.712894 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051534cc-d576-467a-b1cd-059fac48e6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:51.713527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051534cc-d576-467a-b1cd-059fac48e6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:51.713808 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051534cc-d576-467a-b1cd-059fac48e6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:51.714081 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051534cc-d576-467a-b1cd-059fac48e6c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:51.715196 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051534cc-d576-467a-b1cd-059fac48e6c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:51.716636 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051534cc-d576-467a-b1cd-059fac48e6c3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:51.718415 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 104/213] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:22:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:22:51.723587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051534cc-d576-467a-b1cd-059fac48e6c3 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:51.779670 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead93d7-827a-4dcd-b6b4-2181e02f67d8 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:51.812611 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe93696-d20a-4fa6-a524-56ad76cd1c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:51.813234 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe93696-d20a-4fa6-a524-56ad76cd1c74 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:51.813511 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe93696-d20a-4fa6-a524-56ad76cd1c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:51.813787 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe93696-d20a-4fa6-a524-56ad76cd1c74 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:51.814450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe93696-d20a-4fa6-a524-56ad76cd1c74 None admin] RBAC: Adding query filter params (name=member) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:51.814751 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe93696-d20a-4fa6-a524-56ad76cd1c74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:51.816131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe93696-d20a-4fa6-a524-56ad76cd1c74 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:51.822113 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051534cc-d576-467a-b1cd-059fac48e6c3 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:51.823902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 110/214] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 20 13:22:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:22:51.825721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe93696-d20a-4fa6-a524-56ad76cd1c74 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:51.829099 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:51.857130 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076eed1-df95-4a5f-a2af-3184e748a057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:51.857732 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076eed1-df95-4a5f-a2af-3184e748a057 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:51.858011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076eed1-df95-4a5f-a2af-3184e748a057 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:51.858283 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3076eed1-df95-4a5f-a2af-3184e748a057 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:51.858923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076eed1-df95-4a5f-a2af-3184e748a057 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:51.860381 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3076eed1-df95-4a5f-a2af-3184e748a057 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:51.899983 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 105/215] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 20 13:22:51 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:22:51.899983 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076eed1-df95-4a5f-a2af-3184e748a057 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:51.938027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:51.950707 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe93696-d20a-4fa6-a524-56ad76cd1c74 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:51.992930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a09cf-c891-4ad5-a4fb-e86f54e1ab70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:51.993583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a09cf-c891-4ad5-a4fb-e86f54e1ab70 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:51.993863 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a09cf-c891-4ad5-a4fb-e86f54e1ab70 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:51.994135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a09cf-c891-4ad5-a4fb-e86f54e1ab70 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:52.003095 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3076eed1-df95-4a5f-a2af-3184e748a057 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:52.005392 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a09cf-c891-4ad5-a4fb-e86f54e1ab70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:52.011301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-252a09cf-c891-4ad5-a4fb-e86f54e1ab70 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 13:22:52.011301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:52.011301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:52.011301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:52.011301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:52.011301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:52.011301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:52.011301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:52.011301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:52.011301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:52.011301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:52.011301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:52.011301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:52.011301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:52.011301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:52.011301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:52.012444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:52.012444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:52.012444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:52.012444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:52.012444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:52.012444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:52.012444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:52.012444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:52.012444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:52.012444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:52.012444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:52.012444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:52.012444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:52.012444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:52.012444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:52.012444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:52.017941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:52.017941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:52.017941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:52.017941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:52.017941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:52.017941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:52.017941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:52.017941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:52.017941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:52.017941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:52.017941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 13:22:52.017941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:22:52.017941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 111/216] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:22:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:22:52.019877 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a09cf-c891-4ad5-a4fb-e86f54e1ab70 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:52.050646 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99569713-85e4-40a4-b151-60238af2b83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:52.051416 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99569713-85e4-40a4-b151-60238af2b83f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:52.051728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99569713-85e4-40a4-b151-60238af2b83f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:52.052087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99569713-85e4-40a4-b151-60238af2b83f None admin] PATH_INFO: `/v3/users/adb0a4cc5f7b47eeac9ee55c9ecab895` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:52.069684 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99569713-85e4-40a4-b151-60238af2b83f None admin] RBAC: Authorizing `identity:get_user(user_id=adb0a4cc5f7b47eeac9ee55c9ecab895)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:52.071164 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99569713-85e4-40a4-b151-60238af2b83f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:52.077817 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 106/217] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:51 2022] GET /identity/v3/users/adb0a4cc5f7b47eeac9ee55c9ecab895 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:52.079600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99569713-85e4-40a4-b151-60238af2b83f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:52.091255 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:52.137955 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a09cf-c891-4ad5-a4fb-e86f54e1ab70 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:52.183657 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50d0a0-3b34-4faf-9165-d2317dbfadb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:52.184253 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50d0a0-3b34-4faf-9165-d2317dbfadb4 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:52.184474 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50d0a0-3b34-4faf-9165-d2317dbfadb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:52.184749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50d0a0-3b34-4faf-9165-d2317dbfadb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:52.185591 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50d0a0-3b34-4faf-9165-d2317dbfadb4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:52.185790 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50d0a0-3b34-4faf-9165-d2317dbfadb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:52.187233 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50d0a0-3b34-4faf-9165-d2317dbfadb4 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:52.197786 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 112/218] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 20 13:22:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:22:52.204273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:52.206764 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50d0a0-3b34-4faf-9165-d2317dbfadb4 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:52.208305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99569713-85e4-40a4-b151-60238af2b83f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:52.252295 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29c7da-9821-4906-89e4-85cc5aa627b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:52.253393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29c7da-9821-4906-89e4-85cc5aa627b8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:52.253393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29c7da-9821-4906-89e4-85cc5aa627b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:52.253575 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29c7da-9821-4906-89e4-85cc5aa627b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:52.254201 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29c7da-9821-4906-89e4-85cc5aa627b8 None admin] RBAC: Adding query filter params (role.id=5c47f5b95d6f47a78adf8f8e91562e4b, user.id=adb0a4cc5f7b47eeac9ee55c9ecab895, scope.system=all) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:52.254565 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29c7da-9821-4906-89e4-85cc5aa627b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:52.256140 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29c7da-9821-4906-89e4-85cc5aa627b8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:52.264925 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 107/219] 10.222.0.23 () {56 vars in 1308 bytes} [Thu Oct 20 13:22:52 2022] GET /identity/v3/role_assignments?user.id=adb0a4cc5f7b47eeac9ee55c9ecab895&scope.system=all&role.id=5c47f5b95d6f47a78adf8f8e91562e4b => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:22:52.332733 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50d0a0-3b34-4faf-9165-d2317dbfadb4 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:52.363805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:52.374243 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827481a7-5c69-4d6d-9f30-00b9251cba10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:52.375347 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827481a7-5c69-4d6d-9f30-00b9251cba10 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:52.375828 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827481a7-5c69-4d6d-9f30-00b9251cba10 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:52.376321 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827481a7-5c69-4d6d-9f30-00b9251cba10 None admin] PATH_INFO: `/v3/users/8da0b7fb45834462af76a71e2aec7eef` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:52.377799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827481a7-5c69-4d6d-9f30-00b9251cba10 None admin] RBAC: Authorizing `identity:get_user(user_id=8da0b7fb45834462af76a71e2aec7eef)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:52.379505 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827481a7-5c69-4d6d-9f30-00b9251cba10 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:52.381508 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 113/220] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:52 2022] GET /identity/v3/users/8da0b7fb45834462af76a71e2aec7eef => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:52.387260 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29c7da-9821-4906-89e4-85cc5aa627b8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:52.499310 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29c7da-9821-4906-89e4-85cc5aa627b8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:52.537618 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304dc00-2b4e-41eb-8541-fda9191049a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:52.538233 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304dc00-2b4e-41eb-8541-fda9191049a8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:52.538512 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304dc00-2b4e-41eb-8541-fda9191049a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:52.538784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f304dc00-2b4e-41eb-8541-fda9191049a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:52.569687 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:52.610111 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304dc00-2b4e-41eb-8541-fda9191049a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:52.611584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f304dc00-2b4e-41eb-8541-fda9191049a8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:52.721383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 108/221] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:22:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 13:22:52.723656 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827481a7-5c69-4d6d-9f30-00b9251cba10 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:52.833320 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827481a7-5c69-4d6d-9f30-00b9251cba10 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:52.869635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda6b0a-d78a-4955-a697-0fe6e0d6c09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:52.870278 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda6b0a-d78a-4955-a697-0fe6e0d6c09e None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:52.870561 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda6b0a-d78a-4955-a697-0fe6e0d6c09e None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:52.870838 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda6b0a-d78a-4955-a697-0fe6e0d6c09e None admin] PATH_INFO: `/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:52.871963 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda6b0a-d78a-4955-a697-0fe6e0d6c09e None admin] RBAC: Authorizing `identity:get_project(project_id=1fd601dd45764a5ba6b4ac811eaf3eeb)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:52.873466 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda6b0a-d78a-4955-a697-0fe6e0d6c09e None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:52.875415 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 114/222] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:22:52 2022] GET /identity/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:52.881362 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304dc00-2b4e-41eb-8541-fda9191049a8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:52.945579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:52.989335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f304dc00-2b4e-41eb-8541-fda9191049a8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:53.026235 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa03c37-78e7-4460-99f5-91300c6e7671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:53.026867 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa03c37-78e7-4460-99f5-91300c6e7671 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:53.027145 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa03c37-78e7-4460-99f5-91300c6e7671 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:53.027414 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa03c37-78e7-4460-99f5-91300c6e7671 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:53.028165 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa03c37-78e7-4460-99f5-91300c6e7671 None admin] RBAC: Adding query filter params (role.id=83ca567af06444b3b95843cb52d61698, scope.project.id=1fd601dd45764a5ba6b4ac811eaf3eeb, user.id=8da0b7fb45834462af76a71e2aec7eef) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:53.028473 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa03c37-78e7-4460-99f5-91300c6e7671 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:53.030004 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa03c37-78e7-4460-99f5-91300c6e7671 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:53.037419 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 109/223] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:22:52 2022] GET /identity/v3/role_assignments?user.id=8da0b7fb45834462af76a71e2aec7eef&scope.project.id=1fd601dd45764a5ba6b4ac811eaf3eeb&role.id=83ca567af06444b3b95843cb52d61698 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:54.267329 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6733f56-7219-4b64-adb8-73ca5517a603 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:54.267853 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6733f56-7219-4b64-adb8-73ca5517a603 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:54.267926 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6733f56-7219-4b64-adb8-73ca5517a603 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:54.269405 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 115/224] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:54.274484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264fe43-995d-4f08-9a77-dadcde563685 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:54.274867 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264fe43-995d-4f08-9a77-dadcde563685 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:54.275044 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264fe43-995d-4f08-9a77-dadcde563685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:54.293599 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-3264fe43-995d-4f08-9a77-dadcde563685 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:54.294798 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3264fe43-995d-4f08-9a77-dadcde563685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:54.308687 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc2ce6-8e60-4629-aad2-f067721f57e3 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:54.309006 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc2ce6-8e60-4629-aad2-f067721f57e3 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:54.309286 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc2ce6-8e60-4629-aad2-f067721f57e3 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:54.310644 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 116/225] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:22:54.315758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737da51-5664-482e-b274-dd9b8b2e7dbb None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:54.316044 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737da51-5664-482e-b274-dd9b8b2e7dbb None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:54.316324 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737da51-5664-482e-b274-dd9b8b2e7dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:54.338132 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-b737da51-5664-482e-b274-dd9b8b2e7dbb None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:54.339390 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b737da51-5664-482e-b274-dd9b8b2e7dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:54.373055 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 110/226] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:54.405255 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:54.416909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 117/227] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 13:22:54.438237 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182255c0-2f3e-4727-97a6-4f02260891f6 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:54.438524 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182255c0-2f3e-4727-97a6-4f02260891f6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:54.438800 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182255c0-2f3e-4727-97a6-4f02260891f6 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:54.440161 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 111/228] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:54.445959 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d2cc6-6b28-4127-8190-c57485e82e76 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:54.446245 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d2cc6-6b28-4127-8190-c57485e82e76 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:54.446519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d2cc6-6b28-4127-8190-c57485e82e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:54.464430 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-831d2cc6-6b28-4127-8190-c57485e82e76 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:54.465613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-831d2cc6-6b28-4127-8190-c57485e82e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:54.533983 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 118/229] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:54.640196 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182255c0-2f3e-4727-97a6-4f02260891f6 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:54.692566 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d2cc6-6b28-4127-8190-c57485e82e76 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:54.699050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:54.747171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182255c0-2f3e-4727-97a6-4f02260891f6 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:54.787029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf6205-ce5f-41fe-94a0-3c662a8005dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:54.787578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf6205-ce5f-41fe-94a0-3c662a8005dc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:54.787989 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf6205-ce5f-41fe-94a0-3c662a8005dc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:54.788281 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf6205-ce5f-41fe-94a0-3c662a8005dc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:54.797574 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d2cc6-6b28-4127-8190-c57485e82e76 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:54.800968 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf6205-ce5f-41fe-94a0-3c662a8005dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:54.806666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-e4cf6205-ce5f-41fe-94a0-3c662a8005dc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 13:22:54.806666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:54.806666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:54.806666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:54.806666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:54.806666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:54.806666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:54.806666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:54.806666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:54.806666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:54.806666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:54.806666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:54.806666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:54.806666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:54.806666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:54.806666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:54.807584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:54.807584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:54.807584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:54.807584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:54.807584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:54.807584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:54.807584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:54.807584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:54.807584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:54.807584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:54.807584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:54.807584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:54.807584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:54.807584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:54.807584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:54.807584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:54.808536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:54.808536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:54.808536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:54.808536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:54.808536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:54.808536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:54.808536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:54.808536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:54.808536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:54.808536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:54.808536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 13:22:54.808536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:22:54.808536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 112/230] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:22:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:22:54.809441 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf6205-ce5f-41fe-94a0-3c662a8005dc None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:54.823235 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:54.835414 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbad5ff-4af4-4998-94e6-12a8f9f49f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:54.836044 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbad5ff-4af4-4998-94e6-12a8f9f49f41 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:54.836345 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbad5ff-4af4-4998-94e6-12a8f9f49f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:54.836617 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbad5ff-4af4-4998-94e6-12a8f9f49f41 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:54.837737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbad5ff-4af4-4998-94e6-12a8f9f49f41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:54.839176 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbad5ff-4af4-4998-94e6-12a8f9f49f41 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:54.840742 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 119/231] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:22:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:22:54.841803 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbad5ff-4af4-4998-94e6-12a8f9f49f41 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:54.910984 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf6205-ce5f-41fe-94a0-3c662a8005dc None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:54.928464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:54.941006 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbad5ff-4af4-4998-94e6-12a8f9f49f41 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:54.946667 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857945e-c2c4-4dea-b6b6-a6cc842bf622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:54.947300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857945e-c2c4-4dea-b6b6-a6cc842bf622 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:54.947583 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857945e-c2c4-4dea-b6b6-a6cc842bf622 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:54.947860 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857945e-c2c4-4dea-b6b6-a6cc842bf622 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:54.957840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857945e-c2c4-4dea-b6b6-a6cc842bf622 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:54.962926 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-0857945e-c2c4-4dea-b6b6-a6cc842bf622 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:22:54.962926 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:54.962926 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:54.962926 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:54.962926 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:54.962926 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:54.962926 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:54.962926 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:54.962926 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:54.962926 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:54.962926 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:54.962926 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:54.962926 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:54.962926 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:54.962926 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:54.962926 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:54.963957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:54.963957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:54.963957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:54.963957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:54.963957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:54.963957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:54.963957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:54.963957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:54.963957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:54.963957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:54.963957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:54.963957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:54.963957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:54.963957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:54.963957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:54.963957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:54.964775 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:54.964775 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:54.964775 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:54.964775 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:54.964775 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:54.964775 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:54.964775 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:54.964775 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:54.964775 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:54.964775 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:54.964775 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:22:54.964775 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:22:54.964775 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 113/232] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:22:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:22:54.964775 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857945e-c2c4-4dea-b6b6-a6cc842bf622 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:54.980798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818417d-2b3f-40fa-a8a1-fcee7004f932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:54.981442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818417d-2b3f-40fa-a8a1-fcee7004f932 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:54.981721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818417d-2b3f-40fa-a8a1-fcee7004f932 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:54.981989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818417d-2b3f-40fa-a8a1-fcee7004f932 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:54.982651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818417d-2b3f-40fa-a8a1-fcee7004f932 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:54.982955 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818417d-2b3f-40fa-a8a1-fcee7004f932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:54.984336 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818417d-2b3f-40fa-a8a1-fcee7004f932 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:54.992548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 120/233] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 20 13:22:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:54.993354 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818417d-2b3f-40fa-a8a1-fcee7004f932 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:55.045203 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:55.073546 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857945e-c2c4-4dea-b6b6-a6cc842bf622 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:55.094021 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818417d-2b3f-40fa-a8a1-fcee7004f932 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:55.108259 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13093b5c-01a8-4607-a437-3c400282a9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:55.108884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13093b5c-01a8-4607-a437-3c400282a9aa None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:55.109177 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13093b5c-01a8-4607-a437-3c400282a9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:55.109448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13093b5c-01a8-4607-a437-3c400282a9aa None admin] PATH_INFO: `/v3/groups` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:55.110078 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13093b5c-01a8-4607-a437-3c400282a9aa None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:55.111530 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13093b5c-01a8-4607-a437-3c400282a9aa None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:55.127489 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 114/234] 10.222.0.23 () {60 vars in 1134 bytes} [Thu Oct 20 13:22:54 2022] POST /identity/v3/groups => generated 229 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:22:55.127754 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea966709-e645-424e-9b7f-f930128eecc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:55.128364 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea966709-e645-424e-9b7f-f930128eecc5 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:55.128574 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13093b5c-01a8-4607-a437-3c400282a9aa None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:55.128686 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea966709-e645-424e-9b7f-f930128eecc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:55.128936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea966709-e645-424e-9b7f-f930128eecc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:55.129605 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea966709-e645-424e-9b7f-f930128eecc5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:55.129913 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea966709-e645-424e-9b7f-f930128eecc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:55.131417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea966709-e645-424e-9b7f-f930128eecc5 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:55.143720 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 121/235] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 20 13:22:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:55.147649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea966709-e645-424e-9b7f-f930128eecc5 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:55.184237 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:55.263740 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea966709-e645-424e-9b7f-f930128eecc5 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:55.270397 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13093b5c-01a8-4607-a437-3c400282a9aa None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:55.280541 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:55.304981 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfd733-85bc-44d2-8478-e60a8b49cc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:55.305751 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfd733-85bc-44d2-8478-e60a8b49cc3f None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:55.306043 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfd733-85bc-44d2-8478-e60a8b49cc3f None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:55.306332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfd733-85bc-44d2-8478-e60a8b49cc3f None admin] PATH_INFO: `/v3/users/adb0a4cc5f7b47eeac9ee55c9ecab895` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:55.307463 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bfd733-85bc-44d2-8478-e60a8b49cc3f None admin] RBAC: Authorizing `identity:get_user(user_id=adb0a4cc5f7b47eeac9ee55c9ecab895)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:55.308968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bfd733-85bc-44d2-8478-e60a8b49cc3f None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:55.310746 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 122/236] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:55 2022] GET /identity/v3/users/adb0a4cc5f7b47eeac9ee55c9ecab895 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:22:55.316388 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfd733-85bc-44d2-8478-e60a8b49cc3f None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:55.319209 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f810a-d42a-427a-9f3b-bb3d219da114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:55.319794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f810a-d42a-427a-9f3b-bb3d219da114 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:55.320082 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f810a-d42a-427a-9f3b-bb3d219da114 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:55.320358 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f810a-d42a-427a-9f3b-bb3d219da114 None admin] PATH_INFO: `/v3/users/b4361d504a624f06bdf5a766d84d81f1` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:55.335769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f810a-d42a-427a-9f3b-bb3d219da114 None admin] RBAC: Authorizing `identity:get_user(user_id=b4361d504a624f06bdf5a766d84d81f1)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:55.337282 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f810a-d42a-427a-9f3b-bb3d219da114 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:55.339148 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 115/237] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:55 2022] GET /identity/v3/users/b4361d504a624f06bdf5a766d84d81f1 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:55.344705 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f810a-d42a-427a-9f3b-bb3d219da114 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:55.429032 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfd733-85bc-44d2-8478-e60a8b49cc3f None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:55.443059 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:55.450568 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f810a-d42a-427a-9f3b-bb3d219da114 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:55.465279 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a5c0b-22ca-4b39-82ec-48f3546313ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:55.465936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a5c0b-22ca-4b39-82ec-48f3546313ab None admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:55.466218 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a5c0b-22ca-4b39-82ec-48f3546313ab None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:55.466497 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a5c0b-22ca-4b39-82ec-48f3546313ab None admin] PATH_INFO: `/v3/system/users/adb0a4cc5f7b47eeac9ee55c9ecab895/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:55.467897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a5c0b-22ca-4b39-82ec-48f3546313ab None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=adb0a4cc5f7b47eeac9ee55c9ecab895, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:55.469354 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a5c0b-22ca-4b39-82ec-48f3546313ab None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:55.479382 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 123/238] 10.222.0.23 () {58 vars in 1334 bytes} [Thu Oct 20 13:22:55 2022] PUT /identity/v3/system/users/adb0a4cc5f7b47eeac9ee55c9ecab895/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:22:55.505159 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe9822-f831-497b-b167-6833ab4fcf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:55.505159 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe9822-f831-497b-b167-6833ab4fcf6a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:55.505972 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe9822-f831-497b-b167-6833ab4fcf6a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:55.506258 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe9822-f831-497b-b167-6833ab4fcf6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:55.508425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe9822-f831-497b-b167-6833ab4fcf6a None admin] RBAC: Adding query filter params (role.id=83ca567af06444b3b95843cb52d61698, user.id=b4361d504a624f06bdf5a766d84d81f1, scope.system=all) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:55.508425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe9822-f831-497b-b167-6833ab4fcf6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:55.508899 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe9822-f831-497b-b167-6833ab4fcf6a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:55.515655 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 116/239] 10.222.0.23 () {56 vars in 1308 bytes} [Thu Oct 20 13:22:55 2022] GET /identity/v3/role_assignments?user.id=b4361d504a624f06bdf5a766d84d81f1&scope.system=all&role.id=83ca567af06444b3b95843cb52d61698 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:22:57.332075 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52157ce7-42a8-44b7-b8e2-8f585c80748c None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:57.332592 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52157ce7-42a8-44b7-b8e2-8f585c80748c None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:57.332664 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52157ce7-42a8-44b7-b8e2-8f585c80748c None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:57.334676 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 124/240] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:57.339787 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae45271-01f8-4e23-b45d-67c583a0c685 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:57.340158 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae45271-01f8-4e23-b45d-67c583a0c685 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:57.340328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae45271-01f8-4e23-b45d-67c583a0c685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:57.358834 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2ae45271-01f8-4e23-b45d-67c583a0c685 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:57.360010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2ae45271-01f8-4e23-b45d-67c583a0c685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:57.426847 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 117/241] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:57.637535 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c02006-62b4-49e0-8d48-5b2f37250542 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:57.638102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c02006-62b4-49e0-8d48-5b2f37250542 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:57.638273 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c02006-62b4-49e0-8d48-5b2f37250542 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:57.639753 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 125/242] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:22:57.644620 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6a705-ba18-4221-86de-9da9f735a0af None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:57.644935 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6a705-ba18-4221-86de-9da9f735a0af None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:57.645216 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6a705-ba18-4221-86de-9da9f735a0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:57.663558 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-72d6a705-ba18-4221-86de-9da9f735a0af None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:57.664728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-72d6a705-ba18-4221-86de-9da9f735a0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:57.676281 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d2c61-1ca8-4b42-a8d2-02203f0ed5c1 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:57.676567 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d2c61-1ca8-4b42-a8d2-02203f0ed5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:57.676841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d2c61-1ca8-4b42-a8d2-02203f0ed5c1 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:57.679235 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 126/243] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:22:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:22:57.684044 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bc672-cf6c-4017-9e8d-c826a6662234 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:57.684411 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bc672-cf6c-4017-9e8d-c826a6662234 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:57.684599 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bc672-cf6c-4017-9e8d-c826a6662234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:57.702752 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-824bc672-cf6c-4017-9e8d-c826a6662234 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:22:57.703926 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-824bc672-cf6c-4017-9e8d-c826a6662234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:22:57.740969 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 118/244] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:22:57.740969 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6a705-ba18-4221-86de-9da9f735a0af None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:57.746622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:57.760016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:57.780885 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 127/245] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:22:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 13:22:57.851646 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6a705-ba18-4221-86de-9da9f735a0af None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:57.887470 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea618e38-1f7f-46ce-878c-9aea03a9dd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:57.888408 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea618e38-1f7f-46ce-878c-9aea03a9dd3e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:57.888632 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea618e38-1f7f-46ce-878c-9aea03a9dd3e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:57.888922 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea618e38-1f7f-46ce-878c-9aea03a9dd3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:57.890014 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea618e38-1f7f-46ce-878c-9aea03a9dd3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:57.891452 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea618e38-1f7f-46ce-878c-9aea03a9dd3e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:57.893184 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 119/246] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:22:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:57.899030 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824bc672-cf6c-4017-9e8d-c826a6662234 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:58.000200 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824bc672-cf6c-4017-9e8d-c826a6662234 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:58.000384 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea618e38-1f7f-46ce-878c-9aea03a9dd3e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:58.036041 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b043b-35cb-4b52-ac3e-a6d7f8954a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:58.037148 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b043b-35cb-4b52-ac3e-a6d7f8954a86 None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:58.037453 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b043b-35cb-4b52-ac3e-a6d7f8954a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:58.037733 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b043b-35cb-4b52-ac3e-a6d7f8954a86 None admin] PATH_INFO: `/v3/groups` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:58.038368 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b043b-35cb-4b52-ac3e-a6d7f8954a86 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:58.039818 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b043b-35cb-4b52-ac3e-a6d7f8954a86 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:58.056174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 128/247] 10.222.0.23 () {60 vars in 1134 bytes} [Thu Oct 20 13:22:57 2022] POST /identity/v3/groups => generated 226 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:22:58.057015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b043b-35cb-4b52-ac3e-a6d7f8954a86 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:58.099688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:58.101592 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:58.120892 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea618e38-1f7f-46ce-878c-9aea03a9dd3e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:58.160173 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b53d4-2450-4d81-ac39-c0c1a25c6601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:58.160733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b53d4-2450-4d81-ac39-c0c1a25c6601 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:58.161038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b53d4-2450-4d81-ac39-c0c1a25c6601 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:58.161307 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b53d4-2450-4d81-ac39-c0c1a25c6601 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:58.173594 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b53d4-2450-4d81-ac39-c0c1a25c6601 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:58.174856 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b043b-35cb-4b52-ac3e-a6d7f8954a86 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:58.179356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-307b53d4-2450-4d81-ac39-c0c1a25c6601 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 13:22:58.179356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:58.179356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:58.179356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:58.179356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:58.179356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:58.179356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:58.179356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:58.179356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:58.179356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:58.179356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:58.179356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:58.179356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:58.179356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:58.179356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:58.179356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:58.180252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:58.180252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:58.180252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:58.180252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:58.180252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:58.180252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:58.180252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:58.180252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:58.180252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:58.180252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:58.180252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:58.180252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:58.180252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:58.180252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:58.180252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:58.180252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:58.181076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:58.181076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:58.181076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:58.181076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:58.181076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:58.181076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:58.181076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:58.181076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:58.181076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:58.181076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:58.181076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 13:22:58.181076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:22:58.181076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 120/248] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:22:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:22:58.187891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b53d4-2450-4d81-ac39-c0c1a25c6601 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:58.216270 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763734a-221a-4a8c-8c8b-942e330a367d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:58.216955 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763734a-221a-4a8c-8c8b-942e330a367d None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:58.217234 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763734a-221a-4a8c-8c8b-942e330a367d None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:58.217506 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763734a-221a-4a8c-8c8b-942e330a367d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:58.229181 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9763734a-221a-4a8c-8c8b-942e330a367d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:22:58.234470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-9763734a-221a-4a8c-8c8b-942e330a367d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:22:58.234470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:22:58.234470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:22:58.234470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:22:58.234470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:22:58.234470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:22:58.234470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:22:58.234470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:22:58.234470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:22:58.234470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:22:58.234470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:22:58.234470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:22:58.234470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:22:58.234470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:22:58.234470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:22:58.234470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:22:58.235330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:22:58.235330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:22:58.235330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:22:58.235330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:22:58.235330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:22:58.235330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:22:58.235330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:22:58.235330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:22:58.235330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:22:58.235330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:22:58.235330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:22:58.235330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:22:58.235330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:22:58.235330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:22:58.235330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:22:58.235330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:22:58.236131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:22:58.236131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:22:58.236131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:22:58.236131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:22:58.236131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:22:58.236131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:22:58.236131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:22:58.236131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:22:58.236131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:22:58.236131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:22:58.236131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:22:58.236131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:22:58.236131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 129/249] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:22:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:22:58.241848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763734a-221a-4a8c-8c8b-942e330a367d None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:58.315715 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b53d4-2450-4d81-ac39-c0c1a25c6601 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:58.355429 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000553fa-a91d-449e-a4ff-29944d630eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:58.356069 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000553fa-a91d-449e-a4ff-29944d630eab None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:58.356312 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000553fa-a91d-449e-a4ff-29944d630eab None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:58.356583 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000553fa-a91d-449e-a4ff-29944d630eab None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:58.358056 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000553fa-a91d-449e-a4ff-29944d630eab None admin] RBAC: Adding query filter params (name=reader) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:58.358348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000553fa-a91d-449e-a4ff-29944d630eab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:58.359727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000553fa-a91d-449e-a4ff-29944d630eab None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:58.362803 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:58.370304 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 121/250] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 20 13:22:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:22:58.372422 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763734a-221a-4a8c-8c8b-942e330a367d None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:58.376929 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000553fa-a91d-449e-a4ff-29944d630eab None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:58.392980 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:58.414624 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004cb08-7be9-4107-a44f-0d01fafd2172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:58.415205 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004cb08-7be9-4107-a44f-0d01fafd2172 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:58.415424 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004cb08-7be9-4107-a44f-0d01fafd2172 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:58.415690 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004cb08-7be9-4107-a44f-0d01fafd2172 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:58.416359 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004cb08-7be9-4107-a44f-0d01fafd2172 None admin] RBAC: Adding query filter params (name=member) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:58.416661 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004cb08-7be9-4107-a44f-0d01fafd2172 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:58.418114 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004cb08-7be9-4107-a44f-0d01fafd2172 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:58.432950 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 130/251] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 20 13:22:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:22:58.434170 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004cb08-7be9-4107-a44f-0d01fafd2172 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:58.502764 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000553fa-a91d-449e-a4ff-29944d630eab None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:58.551922 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f891b-8055-49a1-a3d3-dc6e40d0451b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:58.553165 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f891b-8055-49a1-a3d3-dc6e40d0451b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:58.553946 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f891b-8055-49a1-a3d3-dc6e40d0451b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:58.554317 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f891b-8055-49a1-a3d3-dc6e40d0451b None admin] PATH_INFO: `/v3/users/b4361d504a624f06bdf5a766d84d81f1` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:58.555565 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976f891b-8055-49a1-a3d3-dc6e40d0451b None admin] RBAC: Authorizing `identity:get_user(user_id=b4361d504a624f06bdf5a766d84d81f1)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:58.557673 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976f891b-8055-49a1-a3d3-dc6e40d0451b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:58.559982 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 122/252] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:58 2022] GET /identity/v3/users/b4361d504a624f06bdf5a766d84d81f1 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:22:58.565689 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f891b-8055-49a1-a3d3-dc6e40d0451b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:58.571201 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004cb08-7be9-4107-a44f-0d01fafd2172 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:58.607460 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61ef4b-8593-4f46-a6ef-fee52a66eff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:58.608096 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61ef4b-8593-4f46-a6ef-fee52a66eff2 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:58.608417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61ef4b-8593-4f46-a6ef-fee52a66eff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:58.608694 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61ef4b-8593-4f46-a6ef-fee52a66eff2 None admin] PATH_INFO: `/v3/users/adb0a4cc5f7b47eeac9ee55c9ecab895` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:58.609855 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61ef4b-8593-4f46-a6ef-fee52a66eff2 None admin] RBAC: Authorizing `identity:get_user(user_id=adb0a4cc5f7b47eeac9ee55c9ecab895)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:58.611336 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61ef4b-8593-4f46-a6ef-fee52a66eff2 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:58.613235 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 131/253] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:22:58 2022] GET /identity/v3/users/adb0a4cc5f7b47eeac9ee55c9ecab895 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:22:58.619712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61ef4b-8593-4f46-a6ef-fee52a66eff2 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:22:58.653327 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:58.667356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:22:58.700556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f891b-8055-49a1-a3d3-dc6e40d0451b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:58.744675 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce14e1a-fc36-45bf-a854-1f992c62e7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:58.745588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce14e1a-fc36-45bf-a854-1f992c62e7fa None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:58.746027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce14e1a-fc36-45bf-a854-1f992c62e7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:58.746370 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61ef4b-8593-4f46-a6ef-fee52a66eff2 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:22:58.746600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce14e1a-fc36-45bf-a854-1f992c62e7fa None admin] PATH_INFO: `/v3/system/users/b4361d504a624f06bdf5a766d84d81f1/roles/83ca567af06444b3b95843cb52d61698` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:58.748264 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce14e1a-fc36-45bf-a854-1f992c62e7fa None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b4361d504a624f06bdf5a766d84d81f1, role_id=83ca567af06444b3b95843cb52d61698)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:58.750262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce14e1a-fc36-45bf-a854-1f992c62e7fa None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:58.762000 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 123/254] 10.222.0.23 () {58 vars in 1334 bytes} [Thu Oct 20 13:22:58 2022] PUT /identity/v3/system/users/b4361d504a624f06bdf5a766d84d81f1/roles/83ca567af06444b3b95843cb52d61698 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:22:58.788515 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ec137-07a3-45fb-b22c-dd50efa71833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:22:58.789286 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ec137-07a3-45fb-b22c-dd50efa71833 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:22:58.789571 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ec137-07a3-45fb-b22c-dd50efa71833 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:22:58.789849 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ec137-07a3-45fb-b22c-dd50efa71833 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:22:58.790590 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ec137-07a3-45fb-b22c-dd50efa71833 None admin] RBAC: Adding query filter params (role.id=5c47f5b95d6f47a78adf8f8e91562e4b, user.id=adb0a4cc5f7b47eeac9ee55c9ecab895, scope.system=all) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:22:58.790902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ec137-07a3-45fb-b22c-dd50efa71833 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:22:58.792390 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ec137-07a3-45fb-b22c-dd50efa71833 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:22:58.807136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 132/255] 10.222.0.23 () {56 vars in 1308 bytes} [Thu Oct 20 13:22:58 2022] GET /identity/v3/role_assignments?user.id=adb0a4cc5f7b47eeac9ee55c9ecab895&scope.system=all&role.id=5c47f5b95d6f47a78adf8f8e91562e4b => generated 510 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:00.777854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49611376-41b1-44f7-8b66-1b93bf719df6 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:00.778419 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49611376-41b1-44f7-8b66-1b93bf719df6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:00.778419 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49611376-41b1-44f7-8b66-1b93bf719df6 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:00.779928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 124/256] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:00.785955 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25091684-2935-4fd0-a6b1-7544325dca18 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:00.787346 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25091684-2935-4fd0-a6b1-7544325dca18 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:00.787831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25091684-2935-4fd0-a6b1-7544325dca18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:00.802910 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce061a8-d7d7-4643-91b9-a0b0312424b1 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:00.803504 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce061a8-d7d7-4643-91b9-a0b0312424b1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:00.803975 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce061a8-d7d7-4643-91b9-a0b0312424b1 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:00.805842 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 125/257] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:00.810839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0052f-48bb-4c4d-9ab7-7623749df0b3 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:00.811502 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0052f-48bb-4c4d-9ab7-7623749df0b3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:00.811980 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0052f-48bb-4c4d-9ab7-7623749df0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:00.813876 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-25091684-2935-4fd0-a6b1-7544325dca18 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:00.816969 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-25091684-2935-4fd0-a6b1-7544325dca18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:00.845888 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-48d0052f-48bb-4c4d-9ab7-7623749df0b3 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:00.847103 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-48d0052f-48bb-4c4d-9ab7-7623749df0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:00.901323 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 133/258] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 20 13:23:00.953400 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 126/259] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 20 13:23:01.447670 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25091684-2935-4fd0-a6b1-7544325dca18 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:01.462059 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25091684-2935-4fd0-a6b1-7544325dca18 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:01.467014 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5397b-f21f-4510-b1aa-9ce8b3b8b36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:01.467937 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5397b-f21f-4510-b1aa-9ce8b3b8b36e admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:01.468406 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5397b-f21f-4510-b1aa-9ce8b3b8b36e admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:01.468895 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5397b-f21f-4510-b1aa-9ce8b3b8b36e admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:01.485277 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5397b-f21f-4510-b1aa-9ce8b3b8b36e admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:01.490916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-69c5397b-f21f-4510-b1aa-9ce8b3b8b36e admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 13:23:01.490916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:01.490916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:01.490916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:01.490916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:01.490916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:01.490916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:01.490916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:01.490916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:01.490916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:01.490916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:01.490916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:01.490916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:01.490916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:01.490916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:01.490916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:01.491799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:01.491799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:01.491799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:01.491799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:01.491799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:01.491799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:01.491799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:01.491799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:01.491799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:01.491799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:01.491799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:01.491799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:01.491799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:01.491799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:01.491799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:01.491799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:01.492568 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:01.492568 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:01.492568 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:01.492568 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:01.492568 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:01.492568 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:01.492568 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:01.492568 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:01.492568 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:01.492568 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:01.492568 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 13:23:01.492568 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:01.500079 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 134/260] 10.222.0.23 () {56 vars in 1135 bytes} [Thu Oct 20 13:23:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 53 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:01.506601 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0052f-48bb-4c4d-9ab7-7623749df0b3 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:01.521589 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5397b-f21f-4510-b1aa-9ce8b3b8b36e admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:01.523603 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0052f-48bb-4c4d-9ab7-7623749df0b3 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:01.532844 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5397b-f21f-4510-b1aa-9ce8b3b8b36e admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:01.534049 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb84fe-a924-4c72-b851-5e56adcfc759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:01.534892 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb84fe-a924-4c72-b851-5e56adcfc759 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:01.535390 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb84fe-a924-4c72-b851-5e56adcfc759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:01.535860 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb84fe-a924-4c72-b851-5e56adcfc759 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:01.537290 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eb84fe-a924-4c72-b851-5e56adcfc759 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:01.537900 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eb84fe-a924-4c72-b851-5e56adcfc759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:01.540060 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: WARNING py.warnings [None req-a4eb84fe-a924-4c72-b851-5e56adcfc759 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 20 13:23:01.540060 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: warnings.warn(msg) Oct 20 13:23:01.540060 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: Oct 20 13:23:01.541425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eb84fe-a924-4c72-b851-5e56adcfc759 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:01.542034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4ac9d-4be0-4537-a2e4-2f18d2fd0a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:01.542675 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4ac9d-4be0-4537-a2e4-2f18d2fd0a29 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:01.542956 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4ac9d-4be0-4537-a2e4-2f18d2fd0a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:01.543225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4ac9d-4be0-4537-a2e4-2f18d2fd0a29 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:01.560042 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f4ac9d-4be0-4537-a2e4-2f18d2fd0a29 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:01.560561 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 127/261] 10.222.0.23 () {56 vars in 1132 bytes} [Thu Oct 20 13:23:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:01.564985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-c9f4ac9d-4be0-4537-a2e4-2f18d2fd0a29 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:23:01.564985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:01.564985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:01.564985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:01.564985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:01.564985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:01.564985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:01.564985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:01.564985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:01.564985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:01.564985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:01.564985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:01.564985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:01.564985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:01.564985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:01.564985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:01.566062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:01.566062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:01.566062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:01.566062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:01.566062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:01.566062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:01.566062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:01.566062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:01.566062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:01.566062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:01.566062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:01.566062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:01.566062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:01.566062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:01.566062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:01.566062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:01.566852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:01.566852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:01.566852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:01.566852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:01.566852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:01.566852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:01.566852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:01.566852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:01.566852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:01.566852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:01.566852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:23:01.566852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:01.567463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb84fe-a924-4c72-b851-5e56adcfc759 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:01.573801 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 135/262] 10.222.0.23 () {56 vars in 1120 bytes} [Thu Oct 20 13:23:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 53 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:01.579268 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4ac9d-4be0-4537-a2e4-2f18d2fd0a29 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:01.582628 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb84fe-a924-4c72-b851-5e56adcfc759 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:01.593194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4ac9d-4be0-4537-a2e4-2f18d2fd0a29 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:01.595058 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fcb83a-8f62-446e-a150-d5bf1f0319d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:01.595912 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcb83a-8f62-446e-a150-d5bf1f0319d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:01.596289 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcb83a-8f62-446e-a150-d5bf1f0319d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:01.596638 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcb83a-8f62-446e-a150-d5bf1f0319d2 admin admin] PATH_INFO: `/v3/groups/c5bb74be25a3401a8e0c3067d4044da1` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:01.602523 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff0478-5088-40b7-8aaa-12f20a59322f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:01.603435 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff0478-5088-40b7-8aaa-12f20a59322f admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:01.603890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff0478-5088-40b7-8aaa-12f20a59322f admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:01.604332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff0478-5088-40b7-8aaa-12f20a59322f admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:01.605230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff0478-5088-40b7-8aaa-12f20a59322f admin admin] RBAC: Adding query filter params (name=member) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:01.605740 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff0478-5088-40b7-8aaa-12f20a59322f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:01.607802 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: WARNING py.warnings [None req-c1ff0478-5088-40b7-8aaa-12f20a59322f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 20 13:23:01.607802 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: warnings.warn(msg) Oct 20 13:23:01.607802 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: Oct 20 13:23:01.609083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff0478-5088-40b7-8aaa-12f20a59322f admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:01.609922 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fcb83a-8f62-446e-a150-d5bf1f0319d2 admin admin] RBAC: Authorizing `identity:get_group(group_id=c5bb74be25a3401a8e0c3067d4044da1)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:01.611061 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: WARNING py.warnings [None req-f4fcb83a-8f62-446e-a150-d5bf1f0319d2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 13:23:01.611061 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: warnings.warn(msg) Oct 20 13:23:01.611061 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: Oct 20 13:23:01.612575 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fcb83a-8f62-446e-a150-d5bf1f0319d2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:01.614463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 128/263] 10.222.0.23 () {56 vars in 1201 bytes} [Thu Oct 20 13:23:01 2022] GET /identity/v3/groups/c5bb74be25a3401a8e0c3067d4044da1 => generated 226 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:01.621526 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:01.625234 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fcb83a-8f62-446e-a150-d5bf1f0319d2 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:01.625884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 136/264] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Oct 20 13:23:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:23:01.634530 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff0478-5088-40b7-8aaa-12f20a59322f admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:01.640318 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fcb83a-8f62-446e-a150-d5bf1f0319d2 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:01.651082 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff0478-5088-40b7-8aaa-12f20a59322f admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:01.656453 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b56875-e3ee-4346-b9e5-3ea7e507aa09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:01.657362 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b56875-e3ee-4346-b9e5-3ea7e507aa09 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:01.657653 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b56875-e3ee-4346-b9e5-3ea7e507aa09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:01.657982 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b56875-e3ee-4346-b9e5-3ea7e507aa09 admin admin] PATH_INFO: `/v3/projects/2a7d45346e944c6089c645b9c6958126` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:01.659005 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffbcc79-dc5c-4e2c-9d8c-7d6e615addaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:01.659226 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b56875-e3ee-4346-b9e5-3ea7e507aa09 admin admin] RBAC: Authorizing `identity:get_project(project_id=2a7d45346e944c6089c645b9c6958126)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:01.660028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbcc79-dc5c-4e2c-9d8c-7d6e615addaa admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:01.660492 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbcc79-dc5c-4e2c-9d8c-7d6e615addaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:01.660968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbcc79-dc5c-4e2c-9d8c-7d6e615addaa admin admin] PATH_INFO: `/v3/groups/c5bb74be25a3401a8e0c3067d4044da1` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:01.662063 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b56875-e3ee-4346-b9e5-3ea7e507aa09 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:01.663010 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffbcc79-dc5c-4e2c-9d8c-7d6e615addaa admin admin] RBAC: Authorizing `identity:get_group(group_id=c5bb74be25a3401a8e0c3067d4044da1)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:01.664213 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: WARNING py.warnings [None req-cffbcc79-dc5c-4e2c-9d8c-7d6e615addaa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 13:23:01.664213 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: warnings.warn(msg) Oct 20 13:23:01.664213 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: Oct 20 13:23:01.664783 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 129/265] 10.222.0.23 () {56 vars in 1207 bytes} [Thu Oct 20 13:23:01 2022] GET /identity/v3/projects/2a7d45346e944c6089c645b9c6958126 => generated 298 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:01.665961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b898ee-fe43-4869-b6e6-4c9eeb4c8fb5 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:01.666192 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffbcc79-dc5c-4e2c-9d8c-7d6e615addaa admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:01.666287 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b898ee-fe43-4869-b6e6-4c9eeb4c8fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:01.666508 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b898ee-fe43-4869-b6e6-4c9eeb4c8fb5 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:01.667761 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 130/266] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:01.668646 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 137/267] 10.222.0.23 () {56 vars in 1201 bytes} [Thu Oct 20 13:23:01 2022] GET /identity/v3/groups/c5bb74be25a3401a8e0c3067d4044da1 => generated 226 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:23:01.679296 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b898ee-fe43-4869-b6e6-4c9eeb4c8fb5 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:01.680946 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffbcc79-dc5c-4e2c-9d8c-7d6e615addaa admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:01.691474 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffbcc79-dc5c-4e2c-9d8c-7d6e615addaa admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:01.701613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4a1f3-3dd6-4967-abef-d3fa9d4728ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:01.702532 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4a1f3-3dd6-4967-abef-d3fa9d4728ea admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:01.703004 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4a1f3-3dd6-4967-abef-d3fa9d4728ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:01.703467 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4a1f3-3dd6-4967-abef-d3fa9d4728ea admin admin] PATH_INFO: `/v3/projects/2a7d45346e944c6089c645b9c6958126` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:01.704771 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c4a1f3-3dd6-4967-abef-d3fa9d4728ea admin admin] RBAC: Authorizing `identity:get_project(project_id=2a7d45346e944c6089c645b9c6958126)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:01.708580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c4a1f3-3dd6-4967-abef-d3fa9d4728ea admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:01.711653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 138/268] 10.222.0.23 () {56 vars in 1207 bytes} [Thu Oct 20 13:23:01 2022] GET /identity/v3/projects/2a7d45346e944c6089c645b9c6958126 => generated 298 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:23:01.712462 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b898ee-fe43-4869-b6e6-4c9eeb4c8fb5 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:01.714266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c445c2e-a7e4-4c01-ba27-a50a24f2b00a None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:01.714609 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c445c2e-a7e4-4c01-ba27-a50a24f2b00a None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:01.718166 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c445c2e-a7e4-4c01-ba27-a50a24f2b00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:01.730089 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05f8bd-7003-4e64-9cac-a386cc804626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:01.731118 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05f8bd-7003-4e64-9cac-a386cc804626 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:01.731236 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05f8bd-7003-4e64-9cac-a386cc804626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:01.731599 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05f8bd-7003-4e64-9cac-a386cc804626 admin admin] PATH_INFO: `/v3/projects/2a7d45346e944c6089c645b9c6958126/groups/c5bb74be25a3401a8e0c3067d4044da1/roles/b0ccf031da9e4a7492c567b607ceb21f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:01.737246 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-8c445c2e-a7e4-4c01-ba27-a50a24f2b00a None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:01.738259 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a05f8bd-7003-4e64-9cac-a386cc804626 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a7d45346e944c6089c645b9c6958126, group_id=c5bb74be25a3401a8e0c3067d4044da1, role_id=b0ccf031da9e4a7492c567b607ceb21f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:01.739027 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-8c445c2e-a7e4-4c01-ba27-a50a24f2b00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:01.739491 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: WARNING py.warnings [None req-4a05f8bd-7003-4e64-9cac-a386cc804626 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 13:23:01.739491 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: warnings.warn(msg) Oct 20 13:23:01.739491 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: Oct 20 13:23:01.741935 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a05f8bd-7003-4e64-9cac-a386cc804626 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:01.769833 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 131/269] 10.222.0.23 () {58 vars in 1463 bytes} [Thu Oct 20 13:23:01 2022] PUT /identity/v3/projects/2a7d45346e944c6089c645b9c6958126/groups/c5bb74be25a3401a8e0c3067d4044da1/roles/b0ccf031da9e4a7492c567b607ceb21f => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:23:01.770780 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05f8bd-7003-4e64-9cac-a386cc804626 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:01.878506 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 139/270] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:01.881517 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4f106-a070-4f51-8eed-4d8939b8deb7 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:01.881896 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4f106-a070-4f51-8eed-4d8939b8deb7 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:01.883529 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b4f106-a070-4f51-8eed-4d8939b8deb7 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:01.885788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 140/271] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:01.892556 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d07471-f5b8-4a89-84a3-51d768e2cd37 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:01.893163 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d07471-f5b8-4a89-84a3-51d768e2cd37 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:01.893643 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d07471-f5b8-4a89-84a3-51d768e2cd37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:01.910402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:01.916090 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05f8bd-7003-4e64-9cac-a386cc804626 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:01.924725 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-97d07471-f5b8-4a89-84a3-51d768e2cd37 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:01.926292 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-97d07471-f5b8-4a89-84a3-51d768e2cd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:01.930108 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80630771-81f8-42ab-8328-e2a41424a48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:01.930795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80630771-81f8-42ab-8328-e2a41424a48c admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:01.931076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80630771-81f8-42ab-8328-e2a41424a48c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:01.931343 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80630771-81f8-42ab-8328-e2a41424a48c admin admin] PATH_INFO: `/v3/projects/2a7d45346e944c6089c645b9c6958126/groups/c5bb74be25a3401a8e0c3067d4044da1/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:01.934715 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80630771-81f8-42ab-8328-e2a41424a48c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a7d45346e944c6089c645b9c6958126, group_id=c5bb74be25a3401a8e0c3067d4044da1, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:01.937631 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80630771-81f8-42ab-8328-e2a41424a48c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:01.942688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 141/272] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 20 13:23:01.967724 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 132/273] 10.222.0.23 () {58 vars in 1463 bytes} [Thu Oct 20 13:23:01 2022] PUT /identity/v3/projects/2a7d45346e944c6089c645b9c6958126/groups/c5bb74be25a3401a8e0c3067d4044da1/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:23:02.241328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d07471-f5b8-4a89-84a3-51d768e2cd37 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:02.251513 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d07471-f5b8-4a89-84a3-51d768e2cd37 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:02.315812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c749b2-3903-4ff9-bb13-7d508bd2591c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:02.316448 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c749b2-3903-4ff9-bb13-7d508bd2591c admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:02.316727 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c749b2-3903-4ff9-bb13-7d508bd2591c admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:02.317026 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c749b2-3903-4ff9-bb13-7d508bd2591c admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:02.331660 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c749b2-3903-4ff9-bb13-7d508bd2591c admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:02.336712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-c0c749b2-3903-4ff9-bb13-7d508bd2591c admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:23:02.336712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:02.336712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:02.336712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:02.336712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:02.336712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:02.336712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:02.336712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:02.336712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:02.336712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:02.336712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:02.336712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:02.336712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:02.336712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:02.336712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:02.336712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:02.337613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:02.337613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:02.337613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:02.337613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:02.337613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:02.337613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:02.337613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:02.337613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:02.337613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:02.337613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:02.337613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:02.337613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:02.337613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:02.337613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:02.337613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:02.337613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:02.338811 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:02.338811 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:02.338811 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:02.338811 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:02.338811 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:02.338811 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:02.338811 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:02.338811 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:02.338811 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:02.338811 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:02.338811 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:23:02.338811 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:02.338811 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 142/274] 10.222.0.23 () {56 vars in 1120 bytes} [Thu Oct 20 13:23:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 97 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:02.346135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80630771-81f8-42ab-8328-e2a41424a48c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:02.357763 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c749b2-3903-4ff9-bb13-7d508bd2591c admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:02.362956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80630771-81f8-42ab-8328-e2a41424a48c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:02.373456 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e2128-ecd9-4193-b9fe-4fa718fe9626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:02.374475 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e2128-ecd9-4193-b9fe-4fa718fe9626 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:02.375654 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e2128-ecd9-4193-b9fe-4fa718fe9626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:02.376145 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e2128-ecd9-4193-b9fe-4fa718fe9626 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:02.377087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e2128-ecd9-4193-b9fe-4fa718fe9626 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:02.377602 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e2128-ecd9-4193-b9fe-4fa718fe9626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:02.379515 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e2128-ecd9-4193-b9fe-4fa718fe9626 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:02.396522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 133/275] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Oct 20 13:23:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:02.403117 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e2128-ecd9-4193-b9fe-4fa718fe9626 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:02.415548 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e2128-ecd9-4193-b9fe-4fa718fe9626 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:02.423485 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91861df0-3304-4dcd-8e86-ba4c14345869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:02.424397 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91861df0-3304-4dcd-8e86-ba4c14345869 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:02.425174 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91861df0-3304-4dcd-8e86-ba4c14345869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:02.425655 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91861df0-3304-4dcd-8e86-ba4c14345869 admin admin] PATH_INFO: `/v3/groups/c5bb74be25a3401a8e0c3067d4044da1` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:02.427037 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91861df0-3304-4dcd-8e86-ba4c14345869 admin admin] RBAC: Authorizing `identity:get_group(group_id=c5bb74be25a3401a8e0c3067d4044da1)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:02.429292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91861df0-3304-4dcd-8e86-ba4c14345869 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:02.432108 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 134/276] 10.222.0.23 () {56 vars in 1201 bytes} [Thu Oct 20 13:23:02 2022] GET /identity/v3/groups/c5bb74be25a3401a8e0c3067d4044da1 => generated 226 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:02.441024 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91861df0-3304-4dcd-8e86-ba4c14345869 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:02.453699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91861df0-3304-4dcd-8e86-ba4c14345869 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:02.466715 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874abc96-5a5b-4d94-875f-e1fc1dd70889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:02.467705 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874abc96-5a5b-4d94-875f-e1fc1dd70889 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:02.468188 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874abc96-5a5b-4d94-875f-e1fc1dd70889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:02.469477 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:02.469726 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874abc96-5a5b-4d94-875f-e1fc1dd70889 admin admin] PATH_INFO: `/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:02.472135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874abc96-5a5b-4d94-875f-e1fc1dd70889 admin admin] RBAC: Authorizing `identity:get_project(project_id=1fd601dd45764a5ba6b4ac811eaf3eeb)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:02.474727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874abc96-5a5b-4d94-875f-e1fc1dd70889 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:02.477578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 135/277] 10.222.0.23 () {56 vars in 1207 bytes} [Thu Oct 20 13:23:02 2022] GET /identity/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb => generated 302 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:02.483860 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874abc96-5a5b-4d94-875f-e1fc1dd70889 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:02.494982 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874abc96-5a5b-4d94-875f-e1fc1dd70889 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:02.504114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23e4de-eb6d-4314-a049-0a2174313281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:02.509647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23e4de-eb6d-4314-a049-0a2174313281 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:02.510224 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23e4de-eb6d-4314-a049-0a2174313281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:02.510768 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23e4de-eb6d-4314-a049-0a2174313281 admin admin] PATH_INFO: `/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb/groups/c5bb74be25a3401a8e0c3067d4044da1/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:02.513748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23e4de-eb6d-4314-a049-0a2174313281 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fd601dd45764a5ba6b4ac811eaf3eeb, group_id=c5bb74be25a3401a8e0c3067d4044da1, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:02.516621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23e4de-eb6d-4314-a049-0a2174313281 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:02.544774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 136/278] 10.222.0.23 () {58 vars in 1463 bytes} [Thu Oct 20 13:23:02 2022] PUT /identity/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb/groups/c5bb74be25a3401a8e0c3067d4044da1/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:23:02.547991 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd679b-5509-4d61-805c-42bbc011b734 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:02.548285 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd679b-5509-4d61-805c-42bbc011b734 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:02.548568 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd679b-5509-4d61-805c-42bbc011b734 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:02.549865 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 137/279] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:02.555670 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd3014-0268-4eba-902e-79b979c851f5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:02.555956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd3014-0268-4eba-902e-79b979c851f5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:02.556236 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd3014-0268-4eba-902e-79b979c851f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:02.576319 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c749b2-3903-4ff9-bb13-7d508bd2591c admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:02.606388 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d3cd3014-0268-4eba-902e-79b979c851f5 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:02.608466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d3cd3014-0268-4eba-902e-79b979c851f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:02.670157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f6878-be1a-4a0e-811f-1452757aa4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:02.670811 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f6878-be1a-4a0e-811f-1452757aa4a3 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:02.671095 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f6878-be1a-4a0e-811f-1452757aa4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:02.671374 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f6878-be1a-4a0e-811f-1452757aa4a3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:02.690096 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f6878-be1a-4a0e-811f-1452757aa4a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:02.696516 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-7c5f6878-be1a-4a0e-811f-1452757aa4a3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 13:23:02.696516 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:02.696516 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:02.696516 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:02.696516 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:02.696516 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:02.696516 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:02.696516 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:02.696516 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:02.696516 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:02.696516 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:02.696516 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:02.696516 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:02.696516 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:02.696516 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:02.696516 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:02.697488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:02.697488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:02.697488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:02.697488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:02.697488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:02.697488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:02.697488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:02.697488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:02.697488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:02.697488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:02.697488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:02.697488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:02.697488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:02.697488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:02.697488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:02.697488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:02.698252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:02.698252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:02.698252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:02.698252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:02.698252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:02.698252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:02.698252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:02.698252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:02.698252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:02.698252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:02.698252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 20 13:23:02.698252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:02.707048 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 143/280] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:23:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:02.708198 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f6878-be1a-4a0e-811f-1452757aa4a3 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:02.749509 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 138/281] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:02 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 20 13:23:02.860844 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f6878-be1a-4a0e-811f-1452757aa4a3 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:02.876643 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:02.908126 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f25b0-fbe5-4cae-9114-8a71b9512999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:02.908792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f25b0-fbe5-4cae-9114-8a71b9512999 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:02.909064 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f25b0-fbe5-4cae-9114-8a71b9512999 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:02.909344 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f25b0-fbe5-4cae-9114-8a71b9512999 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:02.910033 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f25b0-fbe5-4cae-9114-8a71b9512999 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:02.910345 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f25b0-fbe5-4cae-9114-8a71b9512999 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:02.912023 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f25b0-fbe5-4cae-9114-8a71b9512999 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:02.925486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 144/282] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 20 13:23:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:02.931645 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd3014-0268-4eba-902e-79b979c851f5 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:03.041243 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd3014-0268-4eba-902e-79b979c851f5 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:03.050972 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:03.086556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16467579-8f84-4e60-9766-6cda78f51437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:03.087201 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16467579-8f84-4e60-9766-6cda78f51437 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:03.087488 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16467579-8f84-4e60-9766-6cda78f51437 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:03.087768 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16467579-8f84-4e60-9766-6cda78f51437 None admin] PATH_INFO: `/v3/users/b4361d504a624f06bdf5a766d84d81f1` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:03.088951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16467579-8f84-4e60-9766-6cda78f51437 None admin] RBAC: Authorizing `identity:get_user(user_id=b4361d504a624f06bdf5a766d84d81f1)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:03.090412 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16467579-8f84-4e60-9766-6cda78f51437 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:03.092007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 139/283] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:23:02 2022] GET /identity/v3/users/b4361d504a624f06bdf5a766d84d81f1 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:03.097889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f25b0-fbe5-4cae-9114-8a71b9512999 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:03.212705 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f25b0-fbe5-4cae-9114-8a71b9512999 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:03.251363 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6261a3-84eb-4f4d-a7c6-e06d36d9c811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:03.251981 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6261a3-84eb-4f4d-a7c6-e06d36d9c811 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:03.252264 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6261a3-84eb-4f4d-a7c6-e06d36d9c811 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:03.252543 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6261a3-84eb-4f4d-a7c6-e06d36d9c811 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:03.253311 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6261a3-84eb-4f4d-a7c6-e06d36d9c811 None admin] RBAC: Adding query filter params (role.id=83ca567af06444b3b95843cb52d61698, user.id=b4361d504a624f06bdf5a766d84d81f1, scope.system=all) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:03.253626 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6261a3-84eb-4f4d-a7c6-e06d36d9c811 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:03.255593 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6261a3-84eb-4f4d-a7c6-e06d36d9c811 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:03.270539 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 145/284] 10.222.0.23 () {56 vars in 1308 bytes} [Thu Oct 20 13:23:03 2022] GET /identity/v3/role_assignments?user.id=b4361d504a624f06bdf5a766d84d81f1&scope.system=all&role.id=83ca567af06444b3b95843cb52d61698 => generated 510 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:03.338335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16467579-8f84-4e60-9766-6cda78f51437 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:03.350275 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16467579-8f84-4e60-9766-6cda78f51437 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:03.360176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5330758b-2124-4b91-b0af-be945c1badab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:03.360971 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330758b-2124-4b91-b0af-be945c1badab admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:03.361256 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330758b-2124-4b91-b0af-be945c1badab admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:03.361587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330758b-2124-4b91-b0af-be945c1badab admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:03.376312 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5330758b-2124-4b91-b0af-be945c1badab admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:03.381607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-5330758b-2124-4b91-b0af-be945c1badab admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 13:23:03.381607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:03.381607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:03.381607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:03.381607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:03.381607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:03.381607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:03.381607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:03.381607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:03.381607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:03.381607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:03.381607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:03.381607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:03.381607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:03.381607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:03.381607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:03.382448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:03.382448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:03.382448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:03.382448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:03.382448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:03.382448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:03.382448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:03.382448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:03.382448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:03.382448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:03.382448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:03.382448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:03.382448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:03.382448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:03.382448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:03.382448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:03.383218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:03.383218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:03.383218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:03.383218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:03.383218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:03.383218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:03.383218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:03.383218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:03.383218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:03.383218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:03.383218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 13:23:03.383218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:03.383218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 140/285] 10.222.0.23 () {56 vars in 1135 bytes} [Thu Oct 20 13:23:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 45 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:03.390428 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6261a3-84eb-4f4d-a7c6-e06d36d9c811 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:03.405926 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6261a3-84eb-4f4d-a7c6-e06d36d9c811 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:03.412299 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e9959-824f-42b5-9a44-62b4c5f8f4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:03.413085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e9959-824f-42b5-9a44-62b4c5f8f4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:03.413433 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e9959-824f-42b5-9a44-62b4c5f8f4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:03.413589 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e9959-824f-42b5-9a44-62b4c5f8f4b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:03.414263 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e9959-824f-42b5-9a44-62b4c5f8f4b5 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:03.414625 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e9959-824f-42b5-9a44-62b4c5f8f4b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:03.416428 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e9959-824f-42b5-9a44-62b4c5f8f4b5 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:03.430839 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 146/286] 10.222.0.23 () {56 vars in 1132 bytes} [Thu Oct 20 13:23:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:03.436590 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5330758b-2124-4b91-b0af-be945c1badab admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:03.449045 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5330758b-2124-4b91-b0af-be945c1badab admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:03.462127 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5808374-6b5b-45a9-ae3e-a0774b1cedcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:03.462127 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5808374-6b5b-45a9-ae3e-a0774b1cedcf admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:03.462127 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5808374-6b5b-45a9-ae3e-a0774b1cedcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:03.462127 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5808374-6b5b-45a9-ae3e-a0774b1cedcf admin admin] PATH_INFO: `/v3/groups/c5bb74be25a3401a8e0c3067d4044da1` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:03.463470 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5808374-6b5b-45a9-ae3e-a0774b1cedcf admin admin] RBAC: Authorizing `identity:get_group(group_id=c5bb74be25a3401a8e0c3067d4044da1)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:03.465645 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5808374-6b5b-45a9-ae3e-a0774b1cedcf admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:03.467400 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 141/287] 10.222.0.23 () {56 vars in 1201 bytes} [Thu Oct 20 13:23:03 2022] GET /identity/v3/groups/c5bb74be25a3401a8e0c3067d4044da1 => generated 226 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:03.473422 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e9959-824f-42b5-9a44-62b4c5f8f4b5 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:03.485161 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e9959-824f-42b5-9a44-62b4c5f8f4b5 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:03.497518 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc9be6-81ba-4030-bf48-705ee5a2a954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:03.497518 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc9be6-81ba-4030-bf48-705ee5a2a954 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:03.497518 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc9be6-81ba-4030-bf48-705ee5a2a954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:03.497518 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc9be6-81ba-4030-bf48-705ee5a2a954 admin admin] PATH_INFO: `/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:03.497901 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc9be6-81ba-4030-bf48-705ee5a2a954 admin admin] RBAC: Authorizing `identity:get_project(project_id=1fd601dd45764a5ba6b4ac811eaf3eeb)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:03.499711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc9be6-81ba-4030-bf48-705ee5a2a954 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:03.501701 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 147/288] 10.222.0.23 () {56 vars in 1207 bytes} [Thu Oct 20 13:23:03 2022] GET /identity/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb => generated 302 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:03.513195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5808374-6b5b-45a9-ae3e-a0774b1cedcf admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:03.526811 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5808374-6b5b-45a9-ae3e-a0774b1cedcf admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:03.537246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391dd7e-bef8-4c3c-b7be-4304fcdebc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:03.538264 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391dd7e-bef8-4c3c-b7be-4304fcdebc4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:03.538735 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391dd7e-bef8-4c3c-b7be-4304fcdebc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:03.539195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391dd7e-bef8-4c3c-b7be-4304fcdebc4c admin admin] PATH_INFO: `/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb/groups/c5bb74be25a3401a8e0c3067d4044da1/roles/b0ccf031da9e4a7492c567b607ceb21f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:03.546440 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391dd7e-bef8-4c3c-b7be-4304fcdebc4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fd601dd45764a5ba6b4ac811eaf3eeb, group_id=c5bb74be25a3401a8e0c3067d4044da1, role_id=b0ccf031da9e4a7492c567b607ceb21f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:03.549533 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391dd7e-bef8-4c3c-b7be-4304fcdebc4c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:03.576274 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 142/289] 10.222.0.23 () {58 vars in 1463 bytes} [Thu Oct 20 13:23:03 2022] PUT /identity/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb/groups/c5bb74be25a3401a8e0c3067d4044da1/roles/b0ccf031da9e4a7492c567b607ceb21f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:23:04.097318 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee690ecd-8932-428c-80dc-14d5c05853ec None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:04.097841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee690ecd-8932-428c-80dc-14d5c05853ec None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:04.097913 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee690ecd-8932-428c-80dc-14d5c05853ec None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:04.099596 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 148/290] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:04.105473 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c448fa0-253e-4d29-8326-32f3fd7e3ae4 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:04.106311 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c448fa0-253e-4d29-8326-32f3fd7e3ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:04.106774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c448fa0-253e-4d29-8326-32f3fd7e3ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:04.151282 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1c448fa0-253e-4d29-8326-32f3fd7e3ae4 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:04.152816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1c448fa0-253e-4d29-8326-32f3fd7e3ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:04.229126 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 143/291] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 20 13:23:04.778158 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee690ecd-8932-428c-80dc-14d5c05853ec None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:04.794952 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee690ecd-8932-428c-80dc-14d5c05853ec None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:04.800439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040be41-a525-49bd-a2b8-ce8697eeb540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:04.801102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040be41-a525-49bd-a2b8-ce8697eeb540 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:04.801388 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040be41-a525-49bd-a2b8-ce8697eeb540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:04.801660 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040be41-a525-49bd-a2b8-ce8697eeb540 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:04.816855 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040be41-a525-49bd-a2b8-ce8697eeb540 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:04.822156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-f040be41-a525-49bd-a2b8-ce8697eeb540 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:23:04.822156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:04.822156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:04.822156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:04.822156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:04.822156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:04.822156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:04.822156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:04.822156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:04.822156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:04.822156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:04.822156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:04.822156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:04.822156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:04.822156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:04.822156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:04.823001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:04.823001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:04.823001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:04.823001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:04.823001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:04.823001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:04.823001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:04.823001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:04.823001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:04.823001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:04.823001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:04.823001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:04.823001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:04.823001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:04.823001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:04.823001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:04.823768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:04.823768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:04.823768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:04.823768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:04.823768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:04.823768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:04.823768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:04.823768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:04.823768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:04.823768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:04.823768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:23:04.823768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:04.823768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 149/292] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Oct 20 13:23:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 46 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:04.830871 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c448fa0-253e-4d29-8326-32f3fd7e3ae4 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:04.845978 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c448fa0-253e-4d29-8326-32f3fd7e3ae4 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:04.856893 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b645c-f21b-4d7c-b0cd-062303526f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:04.857754 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b645c-f21b-4d7c-b0cd-062303526f6e admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:04.858224 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b645c-f21b-4d7c-b0cd-062303526f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:04.858685 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b645c-f21b-4d7c-b0cd-062303526f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:04.859573 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b645c-f21b-4d7c-b0cd-062303526f6e admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:04.860090 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b645c-f21b-4d7c-b0cd-062303526f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:04.861930 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b645c-f21b-4d7c-b0cd-062303526f6e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:04.876678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 144/293] 10.222.0.23 () {56 vars in 1120 bytes} [Thu Oct 20 13:23:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:04.882812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040be41-a525-49bd-a2b8-ce8697eeb540 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:04.894424 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040be41-a525-49bd-a2b8-ce8697eeb540 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:04.910600 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e3186-7bb5-4e0e-b05d-d1db8999cd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:04.911271 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e3186-7bb5-4e0e-b05d-d1db8999cd1c admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:04.911667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e3186-7bb5-4e0e-b05d-d1db8999cd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:04.911951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e3186-7bb5-4e0e-b05d-d1db8999cd1c admin admin] PATH_INFO: `/v3/groups/3a9a07c9f013429a8ade739c0eb65768` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:04.914497 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:04.919713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2e3186-7bb5-4e0e-b05d-d1db8999cd1c admin admin] RBAC: Authorizing `identity:get_group(group_id=3a9a07c9f013429a8ade739c0eb65768)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:04.921918 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2e3186-7bb5-4e0e-b05d-d1db8999cd1c admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:04.927812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 150/294] 10.222.0.23 () {56 vars in 1201 bytes} [Thu Oct 20 13:23:04 2022] GET /identity/v3/groups/3a9a07c9f013429a8ade739c0eb65768 => generated 229 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:04.933381 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b645c-f21b-4d7c-b0cd-062303526f6e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:04.946618 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b645c-f21b-4d7c-b0cd-062303526f6e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:04.956975 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89786cfa-339c-4ae9-863e-4b9e1390d89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:04.957905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89786cfa-339c-4ae9-863e-4b9e1390d89a admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:04.958912 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89786cfa-339c-4ae9-863e-4b9e1390d89a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:04.959443 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89786cfa-339c-4ae9-863e-4b9e1390d89a admin admin] PATH_INFO: `/v3/projects/c7d49958a3a448d0bed98efad34bc0a2` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:04.965570 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89786cfa-339c-4ae9-863e-4b9e1390d89a admin admin] RBAC: Authorizing `identity:get_project(project_id=c7d49958a3a448d0bed98efad34bc0a2)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:04.967589 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89786cfa-339c-4ae9-863e-4b9e1390d89a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:04.969472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 145/295] 10.222.0.23 () {56 vars in 1207 bytes} [Thu Oct 20 13:23:04 2022] GET /identity/v3/projects/c7d49958a3a448d0bed98efad34bc0a2 => generated 344 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:04.975499 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e3186-7bb5-4e0e-b05d-d1db8999cd1c admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:04.991087 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e3186-7bb5-4e0e-b05d-d1db8999cd1c admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:04.999413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e93c2-f034-4b06-befd-4251779fe488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:05.000147 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e93c2-f034-4b06-befd-4251779fe488 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:05.000437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e93c2-f034-4b06-befd-4251779fe488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:05.000717 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e93c2-f034-4b06-befd-4251779fe488 admin admin] PATH_INFO: `/v3/projects/c7d49958a3a448d0bed98efad34bc0a2/groups/3a9a07c9f013429a8ade739c0eb65768/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:05.002823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e93c2-f034-4b06-befd-4251779fe488 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7d49958a3a448d0bed98efad34bc0a2, group_id=3a9a07c9f013429a8ade739c0eb65768, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:05.003992 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: WARNING py.warnings [None req-e15e93c2-f034-4b06-befd-4251779fe488 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 13:23:05.003992 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: warnings.warn(msg) Oct 20 13:23:05.003992 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: Oct 20 13:23:05.010128 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e93c2-f034-4b06-befd-4251779fe488 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:05.038124 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 151/296] 10.222.0.23 () {58 vars in 1463 bytes} [Thu Oct 20 13:23:04 2022] PUT /identity/v3/projects/c7d49958a3a448d0bed98efad34bc0a2/groups/3a9a07c9f013429a8ade739c0eb65768/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:23:05.040533 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33310248-321d-4908-879d-c4f8c693bfc6 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:05.041052 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33310248-321d-4908-879d-c4f8c693bfc6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:05.041518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33310248-321d-4908-879d-c4f8c693bfc6 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:05.043400 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 146/297] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:05.050898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07dad3-6086-47af-ab8c-1ab7e58709fd None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:05.051199 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07dad3-6086-47af-ab8c-1ab7e58709fd None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:05.051466 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07dad3-6086-47af-ab8c-1ab7e58709fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:05.081292 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-bc07dad3-6086-47af-ab8c-1ab7e58709fd None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:05.082537 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-bc07dad3-6086-47af-ab8c-1ab7e58709fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:05.199418 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 152/298] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:05.284955 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a71ac-1ab1-47ea-ba7d-f1399e7ade16 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:05.285880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a71ac-1ab1-47ea-ba7d-f1399e7ade16 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:05.286890 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a71ac-1ab1-47ea-ba7d-f1399e7ade16 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:05.288925 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 147/299] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:05.294518 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b18bb0-e9ca-4bc2-bfbd-67942985919b None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:05.295037 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b18bb0-e9ca-4bc2-bfbd-67942985919b None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:05.295501 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b18bb0-e9ca-4bc2-bfbd-67942985919b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:05.316308 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-a8b18bb0-e9ca-4bc2-bfbd-67942985919b None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:05.318421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-a8b18bb0-e9ca-4bc2-bfbd-67942985919b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:05.405130 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 153/300] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:05.508672 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a71ac-1ab1-47ea-ba7d-f1399e7ade16 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:05.539846 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdb494-b652-4871-a049-34e46992aeaf None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:05.540441 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdb494-b652-4871-a049-34e46992aeaf None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:05.540923 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdb494-b652-4871-a049-34e46992aeaf None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:05.542822 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 154/301] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:05.548933 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a9230-2587-4e37-82a3-d37b29eaa3e2 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:05.549452 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a9230-2587-4e37-82a3-d37b29eaa3e2 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:05.549942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a9230-2587-4e37-82a3-d37b29eaa3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:05.575003 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-162a9230-2587-4e37-82a3-d37b29eaa3e2 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:05.576469 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-162a9230-2587-4e37-82a3-d37b29eaa3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:05.621075 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:05.648503 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:05.660693 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a71ac-1ab1-47ea-ba7d-f1399e7ade16 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:05.692700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 155/302] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:05.723153 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65814a8f-8919-41b5-89e3-829e8e428c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:05.723824 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65814a8f-8919-41b5-89e3-829e8e428c19 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:05.724136 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65814a8f-8919-41b5-89e3-829e8e428c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:05.724405 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65814a8f-8919-41b5-89e3-829e8e428c19 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:05.742640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65814a8f-8919-41b5-89e3-829e8e428c19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:05.747647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-65814a8f-8919-41b5-89e3-829e8e428c19 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 13:23:05.747647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:05.747647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:05.747647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:05.747647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:05.747647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:05.747647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:05.747647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:05.747647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:05.747647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:05.747647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:05.747647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:05.747647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:05.747647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:05.747647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:05.747647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:05.748682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:05.748682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:05.748682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:05.748682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:05.748682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:05.748682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:05.748682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:05.748682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:05.748682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:05.748682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:05.748682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:05.748682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:05.748682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:05.748682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:05.748682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:05.748682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:05.749633 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:05.749633 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:05.749633 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:05.749633 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:05.749633 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:05.749633 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:05.749633 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:05.749633 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:05.749633 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:05.749633 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:05.749633 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 13:23:05.749633 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:05.749633 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 148/303] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 20 13:23:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:05.754848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a9230-2587-4e37-82a3-d37b29eaa3e2 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:05.835881 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65814a8f-8919-41b5-89e3-829e8e428c19 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:05.931086 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a9230-2587-4e37-82a3-d37b29eaa3e2 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:06.013329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65814a8f-8919-41b5-89e3-829e8e428c19 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:06.023724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5043ee6-af31-43b0-9337-2b5a4e0dadd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:06.024375 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5043ee6-af31-43b0-9337-2b5a4e0dadd9 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:06.024634 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5043ee6-af31-43b0-9337-2b5a4e0dadd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:06.024932 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5043ee6-af31-43b0-9337-2b5a4e0dadd9 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:06.025615 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5043ee6-af31-43b0-9337-2b5a4e0dadd9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:06.025923 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5043ee6-af31-43b0-9337-2b5a4e0dadd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:06.027400 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5043ee6-af31-43b0-9337-2b5a4e0dadd9 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:06.042500 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 156/304] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:06.043343 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5043ee6-af31-43b0-9337-2b5a4e0dadd9 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:06.064369 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b681844-2002-4530-b752-609c075aa12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:06.065159 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b681844-2002-4530-b752-609c075aa12f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:06.065473 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b681844-2002-4530-b752-609c075aa12f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:06.065878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b681844-2002-4530-b752-609c075aa12f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:06.076067 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:06.080681 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b681844-2002-4530-b752-609c075aa12f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:06.088313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-4b681844-2002-4530-b752-609c075aa12f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:23:06.088313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:06.088313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:06.088313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:06.088313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:06.088313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:06.088313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:06.088313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:06.088313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:06.088313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:06.088313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:06.088313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:06.088313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:06.088313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:06.088313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:06.088313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:06.089423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:06.089423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:06.089423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:06.089423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:06.089423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:06.089423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:06.089423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:06.089423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:06.089423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:06.089423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:06.089423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:06.089423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:06.089423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:06.089423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:06.089423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:06.089423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:06.090390 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:06.090390 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:06.090390 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:06.090390 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:06.090390 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:06.090390 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:06.090390 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:06.090390 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:06.090390 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:06.090390 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:06.090390 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:23:06.090390 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:06.091389 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:06.093925 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 149/305] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:23:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:06.094853 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b681844-2002-4530-b752-609c075aa12f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:06.187351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5043ee6-af31-43b0-9337-2b5a4e0dadd9 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:06.226093 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b681844-2002-4530-b752-609c075aa12f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:06.240984 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bcd05-4af3-4d3a-ba7d-3e9432bf3bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:06.242456 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bcd05-4af3-4d3a-ba7d-3e9432bf3bec None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:06.243312 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bcd05-4af3-4d3a-ba7d-3e9432bf3bec None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:06.243727 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bcd05-4af3-4d3a-ba7d-3e9432bf3bec None admin] PATH_INFO: `/v3/roles/member` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:06.258095 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1bcd05-4af3-4d3a-ba7d-3e9432bf3bec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:06.265949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-7c1bcd05-4af3-4d3a-ba7d-3e9432bf3bec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:23:06.265949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:06.265949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:06.265949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:06.265949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:06.265949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:06.265949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:06.265949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:06.265949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:06.265949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:06.265949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:06.265949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:06.265949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:06.265949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:06.265949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:06.265949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:06.266994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:06.266994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:06.266994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:06.266994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:06.266994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:06.266994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:06.266994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:06.266994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:06.266994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:06.266994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:06.266994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:06.266994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:06.266994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:06.266994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:06.266994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:06.266994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:06.267759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:06.267759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:06.267759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:06.267759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:06.267759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:06.267759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:06.267759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:06.267759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:06.267759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:06.267759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:06.267759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 20 13:23:06.267759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:06.269952 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 157/306] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:23:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:06.275702 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67827f33-3deb-4718-9290-13972b49f7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:06.275833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bcd05-4af3-4d3a-ba7d-3e9432bf3bec None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:06.276734 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67827f33-3deb-4718-9290-13972b49f7cf None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:06.277519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67827f33-3deb-4718-9290-13972b49f7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:06.277987 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67827f33-3deb-4718-9290-13972b49f7cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:06.365088 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:06.369640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67827f33-3deb-4718-9290-13972b49f7cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:06.371258 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67827f33-3deb-4718-9290-13972b49f7cf None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:06.406888 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:06.412033 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bcd05-4af3-4d3a-ba7d-3e9432bf3bec None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:06.449265 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd6e06-cba5-488d-b82a-8dd34850f2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:06.450202 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd6e06-cba5-488d-b82a-8dd34850f2c2 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:06.450693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd6e06-cba5-488d-b82a-8dd34850f2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:06.451176 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd6e06-cba5-488d-b82a-8dd34850f2c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:06.452028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd6e06-cba5-488d-b82a-8dd34850f2c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:06.452521 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd6e06-cba5-488d-b82a-8dd34850f2c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:06.454153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd6e06-cba5-488d-b82a-8dd34850f2c2 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:06.464185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 158/307] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 20 13:23:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:06.465312 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd6e06-cba5-488d-b82a-8dd34850f2c2 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:06.490009 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 150/308] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:23:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 13:23:06.495023 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67827f33-3deb-4718-9290-13972b49f7cf None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:06.586254 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd6e06-cba5-488d-b82a-8dd34850f2c2 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:06.613260 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:06.617482 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67827f33-3deb-4718-9290-13972b49f7cf None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:06.630102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592f656-8ed6-48b7-a9bb-7946f5f914c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:06.630787 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592f656-8ed6-48b7-a9bb-7946f5f914c6 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:06.631066 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592f656-8ed6-48b7-a9bb-7946f5f914c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:06.631344 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592f656-8ed6-48b7-a9bb-7946f5f914c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:06.632128 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592f656-8ed6-48b7-a9bb-7946f5f914c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:06.632437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592f656-8ed6-48b7-a9bb-7946f5f914c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:06.634501 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592f656-8ed6-48b7-a9bb-7946f5f914c6 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:06.644987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 159/309] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 20 13:23:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:06.646252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592f656-8ed6-48b7-a9bb-7946f5f914c6 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:06.660824 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2b4cd-341a-47ad-a51d-971211fcfe35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:06.661783 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2b4cd-341a-47ad-a51d-971211fcfe35 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:06.662065 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2b4cd-341a-47ad-a51d-971211fcfe35 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:06.662333 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2b4cd-341a-47ad-a51d-971211fcfe35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:06.676075 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:06.736050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2b4cd-341a-47ad-a51d-971211fcfe35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:06.737531 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2b4cd-341a-47ad-a51d-971211fcfe35 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:06.764957 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592f656-8ed6-48b7-a9bb-7946f5f914c6 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:06.804888 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b87f2-5737-44fc-86a1-3dafef3a8e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:06.806176 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b87f2-5737-44fc-86a1-3dafef3a8e67 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:06.806459 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b87f2-5737-44fc-86a1-3dafef3a8e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:06.806734 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b87f2-5737-44fc-86a1-3dafef3a8e67 None admin] PATH_INFO: `/v3/projects/2a7d45346e944c6089c645b9c6958126` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:06.808029 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b87f2-5737-44fc-86a1-3dafef3a8e67 None admin] RBAC: Authorizing `identity:get_project(project_id=2a7d45346e944c6089c645b9c6958126)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:06.809676 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b87f2-5737-44fc-86a1-3dafef3a8e67 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:06.812918 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 160/310] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:23:06 2022] GET /identity/v3/projects/2a7d45346e944c6089c645b9c6958126 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:06.813829 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b87f2-5737-44fc-86a1-3dafef3a8e67 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:06.845055 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:06.855630 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 151/311] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:23:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 13:23:06.856311 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2b4cd-341a-47ad-a51d-971211fcfe35 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:06.932963 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:06.947429 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b87f2-5737-44fc-86a1-3dafef3a8e67 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:06.982750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2b4cd-341a-47ad-a51d-971211fcfe35 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:06.984126 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37306144-19f5-4b3a-a5fe-fd4c2bab41c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:06.984941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37306144-19f5-4b3a-a5fe-fd4c2bab41c5 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:06.985360 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37306144-19f5-4b3a-a5fe-fd4c2bab41c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:06.985770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37306144-19f5-4b3a-a5fe-fd4c2bab41c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:07.021581 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e682c-efdd-498d-b637-e176548d6896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:07.022213 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e682c-efdd-498d-b637-e176548d6896 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:07.022496 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e682c-efdd-498d-b637-e176548d6896 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:07.022773 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e682c-efdd-498d-b637-e176548d6896 None admin] PATH_INFO: `/v3/groups/c5bb74be25a3401a8e0c3067d4044da1` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:07.026457 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e682c-efdd-498d-b637-e176548d6896 None admin] RBAC: Authorizing `identity:get_group(group_id=c5bb74be25a3401a8e0c3067d4044da1)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:07.027455 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e682c-efdd-498d-b637-e176548d6896 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:07.029340 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 152/312] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Oct 20 13:23:06 2022] GET /identity/v3/groups/c5bb74be25a3401a8e0c3067d4044da1 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:07.029906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e682c-efdd-498d-b637-e176548d6896 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:07.068666 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37306144-19f5-4b3a-a5fe-fd4c2bab41c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:07.070178 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37306144-19f5-4b3a-a5fe-fd4c2bab41c5 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:07.123494 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:07.169725 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e682c-efdd-498d-b637-e176548d6896 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:07.177868 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:07.193062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 161/313] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:23:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 13:23:07.196943 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c42ea-7d6e-4a3b-bf06-9bdd1912b392 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:07.196943 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c42ea-7d6e-4a3b-bf06-9bdd1912b392 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:07.196943 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c42ea-7d6e-4a3b-bf06-9bdd1912b392 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:07.196943 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 162/314] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:07.201083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c42ea-7d6e-4a3b-bf06-9bdd1912b392 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:07.226382 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2895e59-74a3-454c-b8a0-33b524a3cfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:07.227034 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2895e59-74a3-454c-b8a0-33b524a3cfaf None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:07.227313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2895e59-74a3-454c-b8a0-33b524a3cfaf None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:07.227584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2895e59-74a3-454c-b8a0-33b524a3cfaf None admin] PATH_INFO: `/v3/projects/2a7d45346e944c6089c645b9c6958126` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:07.229877 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2895e59-74a3-454c-b8a0-33b524a3cfaf None admin] RBAC: Authorizing `identity:get_project(project_id=2a7d45346e944c6089c645b9c6958126)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:07.231383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2895e59-74a3-454c-b8a0-33b524a3cfaf None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:07.233399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 153/315] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:23:07 2022] GET /identity/v3/projects/2a7d45346e944c6089c645b9c6958126 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:07.233953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2895e59-74a3-454c-b8a0-33b524a3cfaf None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:07.353790 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c42ea-7d6e-4a3b-bf06-9bdd1912b392 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:07.379893 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2895e59-74a3-454c-b8a0-33b524a3cfaf None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:07.402812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a8635-0c66-4c50-8483-7807df937c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:07.403423 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a8635-0c66-4c50-8483-7807df937c29 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:07.403704 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a8635-0c66-4c50-8483-7807df937c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:07.403979 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a8635-0c66-4c50-8483-7807df937c29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:07.404713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a8635-0c66-4c50-8483-7807df937c29 None admin] RBAC: Adding query filter params (group.id=c5bb74be25a3401a8e0c3067d4044da1, role.id=b0ccf031da9e4a7492c567b607ceb21f, scope.project.id=2a7d45346e944c6089c645b9c6958126) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:07.405081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a8635-0c66-4c50-8483-7807df937c29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:07.406578 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a8635-0c66-4c50-8483-7807df937c29 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:07.415447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 163/316] 10.222.0.23 () {56 vars in 1376 bytes} [Thu Oct 20 13:23:07 2022] GET /identity/v3/role_assignments?group.id=c5bb74be25a3401a8e0c3067d4044da1&scope.project.id=2a7d45346e944c6089c645b9c6958126&role.id=b0ccf031da9e4a7492c567b607ceb21f => generated 611 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:07.416819 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689f95b-0428-4106-b44e-04382957cebd None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:07.417230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689f95b-0428-4106-b44e-04382957cebd None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:07.417430 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689f95b-0428-4106-b44e-04382957cebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:07.427051 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ecbd2-98e7-45a5-b210-4c9422a987b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:07.427870 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ecbd2-98e7-45a5-b210-4c9422a987b7 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:07.427964 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ecbd2-98e7-45a5-b210-4c9422a987b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:07.428217 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ecbd2-98e7-45a5-b210-4c9422a987b7 None admin] PATH_INFO: `/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:07.429476 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ecbd2-98e7-45a5-b210-4c9422a987b7 None admin] RBAC: Authorizing `identity:get_project(project_id=1fd601dd45764a5ba6b4ac811eaf3eeb)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:07.431014 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ecbd2-98e7-45a5-b210-4c9422a987b7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:07.433108 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 154/317] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:23:07 2022] GET /identity/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:07.438815 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ecbd2-98e7-45a5-b210-4c9422a987b7 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:07.439286 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-f689f95b-0428-4106-b44e-04382957cebd None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:07.440826 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-f689f95b-0428-4106-b44e-04382957cebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:07.443829 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:07.520902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:07.548386 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 164/318] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:07.549474 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689f95b-0428-4106-b44e-04382957cebd None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:07.621568 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ecbd2-98e7-45a5-b210-4c9422a987b7 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:07.666855 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce6c52-70ea-447d-816e-90b5fa1454f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:07.667506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce6c52-70ea-447d-816e-90b5fa1454f1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:07.667788 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce6c52-70ea-447d-816e-90b5fa1454f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:07.668063 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce6c52-70ea-447d-816e-90b5fa1454f1 None admin] PATH_INFO: `/v3/groups/c5bb74be25a3401a8e0c3067d4044da1` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:07.669241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce6c52-70ea-447d-816e-90b5fa1454f1 None admin] RBAC: Authorizing `identity:get_group(group_id=c5bb74be25a3401a8e0c3067d4044da1)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:07.670784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce6c52-70ea-447d-816e-90b5fa1454f1 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:07.672426 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 155/319] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Oct 20 13:23:07 2022] GET /identity/v3/groups/c5bb74be25a3401a8e0c3067d4044da1 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:07.674767 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97b1df-d3d6-4e63-ba91-d8136fb47155 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:07.675053 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97b1df-d3d6-4e63-ba91-d8136fb47155 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:07.675329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97b1df-d3d6-4e63-ba91-d8136fb47155 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:07.676652 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 156/320] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:07.683223 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235951a-55e0-4f16-88db-6c6643a56a35 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:07.683509 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235951a-55e0-4f16-88db-6c6643a56a35 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:07.683784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235951a-55e0-4f16-88db-6c6643a56a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:07.701051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689f95b-0428-4106-b44e-04382957cebd None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:07.704899 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-4235951a-55e0-4f16-88db-6c6643a56a35 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:07.706107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4235951a-55e0-4f16-88db-6c6643a56a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:07.749364 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae5890-08eb-44dc-b240-54f125d0bb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:07.750117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae5890-08eb-44dc-b240-54f125d0bb4a None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:07.750398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae5890-08eb-44dc-b240-54f125d0bb4a None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:07.750671 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae5890-08eb-44dc-b240-54f125d0bb4a None admin] PATH_INFO: `/v3/groups/c5bb74be25a3401a8e0c3067d4044da1` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:07.751831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae5890-08eb-44dc-b240-54f125d0bb4a None admin] RBAC: Authorizing `identity:get_group(group_id=c5bb74be25a3401a8e0c3067d4044da1)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:07.753509 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae5890-08eb-44dc-b240-54f125d0bb4a None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:07.760371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 165/321] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Oct 20 13:23:07 2022] GET /identity/v3/groups/c5bb74be25a3401a8e0c3067d4044da1 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:07.761316 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae5890-08eb-44dc-b240-54f125d0bb4a None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:07.798062 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 157/322] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:07.799274 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235951a-55e0-4f16-88db-6c6643a56a35 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:07.804917 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:07.826042 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:07.889383 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae5890-08eb-44dc-b240-54f125d0bb4a None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:07.929162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d808e-f37e-468b-aa62-28c1923d16c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:07.929998 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d808e-f37e-468b-aa62-28c1923d16c0 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:07.930456 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d808e-f37e-468b-aa62-28c1923d16c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:07.930862 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d808e-f37e-468b-aa62-28c1923d16c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:07.931824 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d808e-f37e-468b-aa62-28c1923d16c0 None admin] RBAC: Adding query filter params (group.id=c5bb74be25a3401a8e0c3067d4044da1, role.id=5c47f5b95d6f47a78adf8f8e91562e4b, scope.project.id=2a7d45346e944c6089c645b9c6958126) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:07.932397 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d808e-f37e-468b-aa62-28c1923d16c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:07.934066 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d808e-f37e-468b-aa62-28c1923d16c0 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:07.940466 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 166/323] 10.222.0.23 () {56 vars in 1376 bytes} [Thu Oct 20 13:23:07 2022] GET /identity/v3/role_assignments?group.id=c5bb74be25a3401a8e0c3067d4044da1&scope.project.id=2a7d45346e944c6089c645b9c6958126&role.id=5c47f5b95d6f47a78adf8f8e91562e4b => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:07.941602 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d808e-f37e-468b-aa62-28c1923d16c0 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:07.941934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235951a-55e0-4f16-88db-6c6643a56a35 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:07.997486 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f73f5bb-2bc7-444e-b943-4893e5e560c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:07.998016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f73f5bb-2bc7-444e-b943-4893e5e560c1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:07.998822 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f73f5bb-2bc7-444e-b943-4893e5e560c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:07.999113 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f73f5bb-2bc7-444e-b943-4893e5e560c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:07.999871 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f73f5bb-2bc7-444e-b943-4893e5e560c1 None admin] RBAC: Adding query filter params (group.id=c5bb74be25a3401a8e0c3067d4044da1, role.id=5c47f5b95d6f47a78adf8f8e91562e4b, scope.project.id=1fd601dd45764a5ba6b4ac811eaf3eeb) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:08.000190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f73f5bb-2bc7-444e-b943-4893e5e560c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:08.002472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f73f5bb-2bc7-444e-b943-4893e5e560c1 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:08.021161 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 158/324] 10.222.0.23 () {56 vars in 1376 bytes} [Thu Oct 20 13:23:07 2022] GET /identity/v3/role_assignments?group.id=c5bb74be25a3401a8e0c3067d4044da1&scope.project.id=1fd601dd45764a5ba6b4ac811eaf3eeb&role.id=5c47f5b95d6f47a78adf8f8e91562e4b => generated 611 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:08.075368 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d808e-f37e-468b-aa62-28c1923d16c0 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:08.097059 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:08.112916 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f73f5bb-2bc7-444e-b943-4893e5e560c1 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:08.124670 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a2f297-8e25-438d-a683-7ebf2b4d74f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:08.125441 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2f297-8e25-438d-a683-7ebf2b4d74f3 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:08.125728 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2f297-8e25-438d-a683-7ebf2b4d74f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:08.126006 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a2f297-8e25-438d-a683-7ebf2b4d74f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:08.143151 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a2f297-8e25-438d-a683-7ebf2b4d74f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:08.151119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-c8a2f297-8e25-438d-a683-7ebf2b4d74f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 13:23:08.151119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:08.151119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:08.151119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:08.151119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:08.151119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:08.151119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:08.151119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:08.151119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:08.151119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:08.151119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:08.151119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:08.151119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:08.151119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:08.151119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:08.151119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:08.156995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:08.156995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:08.156995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:08.156995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:08.156995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:08.156995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:08.156995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:08.156995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:08.156995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:08.156995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:08.156995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:08.156995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:08.156995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:08.156995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:08.156995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:08.156995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:08.158758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:08.158758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:08.158758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:08.158758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:08.158758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:08.158758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:08.158758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:08.158758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:08.158758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:08.158758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:08.158758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 13:23:08.158758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:08.158758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 167/325] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 20 13:23:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:08.160362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a2f297-8e25-438d-a683-7ebf2b4d74f3 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:08.228805 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:08.278637 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f73f5bb-2bc7-444e-b943-4893e5e560c1 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:08.303475 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a2f297-8e25-438d-a683-7ebf2b4d74f3 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:08.340262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25af74-b007-4318-9324-2720df6538cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:08.340262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25af74-b007-4318-9324-2720df6538cb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:08.340262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25af74-b007-4318-9324-2720df6538cb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:08.340262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25af74-b007-4318-9324-2720df6538cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:08.352558 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25af74-b007-4318-9324-2720df6538cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:08.354201 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c1414e-97c3-4023-b41d-b3a3100d9bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:08.355283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c1414e-97c3-4023-b41d-b3a3100d9bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:08.355714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c1414e-97c3-4023-b41d-b3a3100d9bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:08.355985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c1414e-97c3-4023-b41d-b3a3100d9bf1 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:08.356651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c1414e-97c3-4023-b41d-b3a3100d9bf1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:08.356990 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c1414e-97c3-4023-b41d-b3a3100d9bf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:08.357739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-2b25af74-b007-4318-9324-2720df6538cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:23:08.357739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:08.357739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:08.357739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:08.357739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:08.357739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:08.357739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:08.357739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:08.357739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:08.357739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:08.357739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:08.357739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:08.357739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:08.357739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:08.357739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:08.357739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:08.358745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:08.358745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:08.358745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:08.358745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:08.358745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:08.358745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:08.358745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:08.358745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:08.358745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:08.358745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:08.358745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:08.358745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:08.358745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:08.358745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:08.358745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:08.358745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:08.359620 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:08.359620 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:08.359620 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:08.359620 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:08.359620 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:08.359620 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:08.359620 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:08.359620 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:08.359620 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:08.359620 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:08.359620 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:23:08.359620 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:08.360354 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c1414e-97c3-4023-b41d-b3a3100d9bf1 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:08.360430 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 159/326] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 20 13:23:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:23:08.365648 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25af74-b007-4318-9324-2720df6538cb None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:08.369710 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 168/327] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:08.375565 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c1414e-97c3-4023-b41d-b3a3100d9bf1 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:08.421544 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:08.486312 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25af74-b007-4318-9324-2720df6538cb None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:08.492234 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c1414e-97c3-4023-b41d-b3a3100d9bf1 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:08.523334 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:08.534639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b271205-c485-4243-9a97-4c759256ee0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:08.535468 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b271205-c485-4243-9a97-4c759256ee0d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:08.535746 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b271205-c485-4243-9a97-4c759256ee0d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:08.536016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b271205-c485-4243-9a97-4c759256ee0d None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:08.537319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b271205-c485-4243-9a97-4c759256ee0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:08.537622 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b271205-c485-4243-9a97-4c759256ee0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:08.538145 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e144b-335b-4c23-b146-f85ae076478b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:08.538758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e144b-335b-4c23-b146-f85ae076478b None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:08.538991 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b271205-c485-4243-9a97-4c759256ee0d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:08.539093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e144b-335b-4c23-b146-f85ae076478b None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:08.539301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e144b-335b-4c23-b146-f85ae076478b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:08.548168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 160/328] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:23:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:23:08.554996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b271205-c485-4243-9a97-4c759256ee0d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:08.617344 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e144b-335b-4c23-b146-f85ae076478b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:08.619228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e144b-335b-4c23-b146-f85ae076478b None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:08.664931 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:08.676383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b271205-c485-4243-9a97-4c759256ee0d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:08.713633 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769b8ae-99d6-44d8-a3b2-b3249b4159a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:08.714253 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769b8ae-99d6-44d8-a3b2-b3249b4159a3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:08.714530 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769b8ae-99d6-44d8-a3b2-b3249b4159a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:08.714795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769b8ae-99d6-44d8-a3b2-b3249b4159a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:08.733247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 169/329] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:23:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 20 13:23:08.738757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e144b-335b-4c23-b146-f85ae076478b None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:08.780319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:08.801605 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769b8ae-99d6-44d8-a3b2-b3249b4159a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:08.803981 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769b8ae-99d6-44d8-a3b2-b3249b4159a3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:08.858352 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e144b-335b-4c23-b146-f85ae076478b None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:08.893993 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbcf08e-c61b-4fd8-bf0d-1a85589ff5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:08.894623 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcf08e-c61b-4fd8-bf0d-1a85589ff5b0 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:08.894901 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcf08e-c61b-4fd8-bf0d-1a85589ff5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:08.895171 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcf08e-c61b-4fd8-bf0d-1a85589ff5b0 None admin] PATH_INFO: `/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:08.896791 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbcf08e-c61b-4fd8-bf0d-1a85589ff5b0 None admin] RBAC: Authorizing `identity:get_project(project_id=1fd601dd45764a5ba6b4ac811eaf3eeb)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:08.898266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbcf08e-c61b-4fd8-bf0d-1a85589ff5b0 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:08.900029 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 170/330] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:23:08 2022] GET /identity/v3/projects/1fd601dd45764a5ba6b4ac811eaf3eeb => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:08.905374 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbcf08e-c61b-4fd8-bf0d-1a85589ff5b0 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:08.913499 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:08.916637 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 161/331] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:23:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 13:23:08.922166 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769b8ae-99d6-44d8-a3b2-b3249b4159a3 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:09.003113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbcf08e-c61b-4fd8-bf0d-1a85589ff5b0 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:09.006736 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:09.019030 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769b8ae-99d6-44d8-a3b2-b3249b4159a3 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:09.035880 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e857227-7ba1-46f5-8979-ab3c940f7355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:09.037252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e857227-7ba1-46f5-8979-ab3c940f7355 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:09.037488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e857227-7ba1-46f5-8979-ab3c940f7355 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:09.037756 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e857227-7ba1-46f5-8979-ab3c940f7355 None admin] PATH_INFO: `/v3/groups/c5bb74be25a3401a8e0c3067d4044da1` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:09.038847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e857227-7ba1-46f5-8979-ab3c940f7355 None admin] RBAC: Authorizing `identity:get_group(group_id=c5bb74be25a3401a8e0c3067d4044da1)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:09.040260 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e857227-7ba1-46f5-8979-ab3c940f7355 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:09.041845 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 171/332] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Oct 20 13:23:08 2022] GET /identity/v3/groups/c5bb74be25a3401a8e0c3067d4044da1 => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:23:09.047545 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e857227-7ba1-46f5-8979-ab3c940f7355 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:09.053055 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a47677-168b-42ad-8b85-3c55a8c05755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:09.053688 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a47677-168b-42ad-8b85-3c55a8c05755 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:09.053962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a47677-168b-42ad-8b85-3c55a8c05755 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:09.054229 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a47677-168b-42ad-8b85-3c55a8c05755 None admin] PATH_INFO: `/v3/projects/c7d49958a3a448d0bed98efad34bc0a2` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:09.055318 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a47677-168b-42ad-8b85-3c55a8c05755 None admin] RBAC: Authorizing `identity:get_project(project_id=c7d49958a3a448d0bed98efad34bc0a2)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:09.056742 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a47677-168b-42ad-8b85-3c55a8c05755 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:09.058380 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 162/333] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:23:08 2022] GET /identity/v3/projects/c7d49958a3a448d0bed98efad34bc0a2 => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:23:09.063605 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a47677-168b-42ad-8b85-3c55a8c05755 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:09.125872 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:09.150434 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e857227-7ba1-46f5-8979-ab3c940f7355 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:09.160856 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a47677-168b-42ad-8b85-3c55a8c05755 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:09.183972 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ff770-7fc6-4f4c-bbba-c8e18dac4747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:09.184571 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ff770-7fc6-4f4c-bbba-c8e18dac4747 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:09.184852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ff770-7fc6-4f4c-bbba-c8e18dac4747 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:09.185158 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ff770-7fc6-4f4c-bbba-c8e18dac4747 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:09.185882 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ff770-7fc6-4f4c-bbba-c8e18dac4747 None admin] RBAC: Adding query filter params (group.id=c5bb74be25a3401a8e0c3067d4044da1, role.id=b0ccf031da9e4a7492c567b607ceb21f, scope.project.id=1fd601dd45764a5ba6b4ac811eaf3eeb) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:09.186470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ff770-7fc6-4f4c-bbba-c8e18dac4747 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:09.187935 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ff770-7fc6-4f4c-bbba-c8e18dac4747 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:09.193966 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 172/334] 10.222.0.23 () {56 vars in 1376 bytes} [Thu Oct 20 13:23:09 2022] GET /identity/v3/role_assignments?group.id=c5bb74be25a3401a8e0c3067d4044da1&scope.project.id=1fd601dd45764a5ba6b4ac811eaf3eeb&role.id=b0ccf031da9e4a7492c567b607ceb21f => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:23:09.195726 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3e46c-8340-4246-99f5-d700c5831837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:09.196355 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3e46c-8340-4246-99f5-d700c5831837 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:09.196630 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3e46c-8340-4246-99f5-d700c5831837 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:09.196923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3e46c-8340-4246-99f5-d700c5831837 None admin] PATH_INFO: `/v3/groups/3a9a07c9f013429a8ade739c0eb65768` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:09.198045 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a3e46c-8340-4246-99f5-d700c5831837 None admin] RBAC: Authorizing `identity:get_group(group_id=3a9a07c9f013429a8ade739c0eb65768)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:09.199487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a3e46c-8340-4246-99f5-d700c5831837 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:09.201154 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 163/335] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Oct 20 13:23:09 2022] GET /identity/v3/groups/3a9a07c9f013429a8ade739c0eb65768 => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:23:09.207311 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ff770-7fc6-4f4c-bbba-c8e18dac4747 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:09.310276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ff770-7fc6-4f4c-bbba-c8e18dac4747 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:09.349996 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2281249-7ffd-4538-b0c3-164c955368c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:09.350614 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2281249-7ffd-4538-b0c3-164c955368c1 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:09.350891 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2281249-7ffd-4538-b0c3-164c955368c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:09.351155 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2281249-7ffd-4538-b0c3-164c955368c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:09.351891 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2281249-7ffd-4538-b0c3-164c955368c1 None admin] RBAC: Adding query filter params (group.id=3a9a07c9f013429a8ade739c0eb65768, role.id=2bcf35e1bc5f4634b0aa6411fad2f43f, scope.project.id=c7d49958a3a448d0bed98efad34bc0a2) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:09.352195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2281249-7ffd-4538-b0c3-164c955368c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:09.353674 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2281249-7ffd-4538-b0c3-164c955368c1 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:09.359893 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 173/336] 10.222.0.23 () {56 vars in 1376 bytes} [Thu Oct 20 13:23:09 2022] GET /identity/v3/role_assignments?group.id=3a9a07c9f013429a8ade739c0eb65768&scope.project.id=c7d49958a3a448d0bed98efad34bc0a2&role.id=2bcf35e1bc5f4634b0aa6411fad2f43f => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:11.902172 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521417e-6f04-443b-bb8e-6032881b4bae None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:11.903191 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521417e-6f04-443b-bb8e-6032881b4bae None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:11.903692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521417e-6f04-443b-bb8e-6032881b4bae None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:11.905525 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 164/337] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:11.911759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567a43c-481d-4d96-aa70-667c381e9ce9 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:11.912609 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567a43c-481d-4d96-aa70-667c381e9ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:11.913133 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567a43c-481d-4d96-aa70-667c381e9ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:11.926614 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:11.941858 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-b567a43c-481d-4d96-aa70-667c381e9ce9 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:11.943590 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b567a43c-481d-4d96-aa70-667c381e9ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:12.045330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 174/338] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 13:23:12.170509 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450ed0d-1cb3-4361-b130-a35e65c3e274 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:12.171113 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450ed0d-1cb3-4361-b130-a35e65c3e274 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:12.171594 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450ed0d-1cb3-4361-b130-a35e65c3e274 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:12.173472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 165/339] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:12.178915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70010e-ac3a-4980-bac3-5d7201a62023 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:12.179439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70010e-ac3a-4980-bac3-5d7201a62023 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:12.179947 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70010e-ac3a-4980-bac3-5d7201a62023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:12.207594 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-3e70010e-ac3a-4980-bac3-5d7201a62023 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:12.209741 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-3e70010e-ac3a-4980-bac3-5d7201a62023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:12.302770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 175/340] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:12.428730 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1c47f-e074-444b-97e5-711e0daaf7f0 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:12.429410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1c47f-e074-444b-97e5-711e0daaf7f0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:12.429908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1c47f-e074-444b-97e5-711e0daaf7f0 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:12.431671 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 166/341] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:12.437440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147d426-7430-434b-ab80-ab8207eb4168 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:12.437954 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147d426-7430-434b-ab80-ab8207eb4168 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:12.438449 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147d426-7430-434b-ab80-ab8207eb4168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:12.443146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1c47f-e074-444b-97e5-711e0daaf7f0 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:12.463426 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-f147d426-7430-434b-ab80-ab8207eb4168 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:12.464629 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-f147d426-7430-434b-ab80-ab8207eb4168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:12.471547 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:12.576843 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1c47f-e074-444b-97e5-711e0daaf7f0 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:12.579257 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:12.600928 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 176/342] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:12.629398 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e353d4-ea6b-4dd4-8fc5-2f899045ceb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:12.630588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e353d4-ea6b-4dd4-8fc5-2f899045ceb4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:12.630876 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e353d4-ea6b-4dd4-8fc5-2f899045ceb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:12.631153 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e353d4-ea6b-4dd4-8fc5-2f899045ceb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:12.649170 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e353d4-ea6b-4dd4-8fc5-2f899045ceb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:12.650635 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e353d4-ea6b-4dd4-8fc5-2f899045ceb4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:12.665707 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-78e353d4-ea6b-4dd4-8fc5-2f899045ceb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:12.665707 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:12.665707 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:12.665707 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:12.665707 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:12.665707 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:12.665707 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:12.665707 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:12.665707 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:12.665707 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:12.665707 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:12.665707 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:12.665707 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:12.665707 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:12.665707 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:12.665707 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:12.666651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:12.666651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:12.666651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:12.666651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:12.666651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:12.666651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:12.666651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:12.666651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:12.666651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:12.666651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:12.666651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:12.666651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:12.666651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:12.666651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:12.666651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:12.666651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:12.666651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:12.667502 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:12.667502 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:12.667502 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:12.671744 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 167/343] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:12.672755 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147d426-7430-434b-ab80-ab8207eb4168 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:12.674324 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e353d4-ea6b-4dd4-8fc5-2f899045ceb4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:12.758184 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:12.797138 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e353d4-ea6b-4dd4-8fc5-2f899045ceb4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:12.838653 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad1292-4005-414a-800e-09244e210b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:12.839294 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad1292-4005-414a-800e-09244e210b34 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:12.839577 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad1292-4005-414a-800e-09244e210b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:12.839849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad1292-4005-414a-800e-09244e210b34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:12.856826 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad1292-4005-414a-800e-09244e210b34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:12.859073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad1292-4005-414a-800e-09244e210b34 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:12.867037 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147d426-7430-434b-ab80-ab8207eb4168 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:12.886484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-66ad1292-4005-414a-800e-09244e210b34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:12.886484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:12.886484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:12.886484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:12.886484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:12.886484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:12.886484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:12.886484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:12.886484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:12.886484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:12.886484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:12.886484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:12.886484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:12.886484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:12.886484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:12.886484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:12.887432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:12.887432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:12.887432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:12.887432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:12.887432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:12.887432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:12.887432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:12.887432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:12.887432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:12.887432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:12.887432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:12.887432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:12.887432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:12.887432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:12.887432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:12.887432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:12.887432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:12.888533 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:12.888533 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:12.888533 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:12.890212 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 168/344] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:12.895459 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad1292-4005-414a-800e-09244e210b34 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:12.935306 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41103e4-19ae-4ee2-8580-a8ffcf8f0aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:12.935922 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41103e4-19ae-4ee2-8580-a8ffcf8f0aea None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:12.936414 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41103e4-19ae-4ee2-8580-a8ffcf8f0aea None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:12.936930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41103e4-19ae-4ee2-8580-a8ffcf8f0aea None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:12.937788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41103e4-19ae-4ee2-8580-a8ffcf8f0aea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:12.938341 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41103e4-19ae-4ee2-8580-a8ffcf8f0aea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:12.939987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41103e4-19ae-4ee2-8580-a8ffcf8f0aea None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:12.952773 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 177/345] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:12.954012 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41103e4-19ae-4ee2-8580-a8ffcf8f0aea None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:13.019902 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad1292-4005-414a-800e-09244e210b34 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:13.029029 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:13.048591 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:13.058799 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c676197-deff-4258-9ff9-56b4bb09bb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:13.059406 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c676197-deff-4258-9ff9-56b4bb09bb06 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:13.059684 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c676197-deff-4258-9ff9-56b4bb09bb06 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:13.059954 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c676197-deff-4258-9ff9-56b4bb09bb06 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:13.060591 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c676197-deff-4258-9ff9-56b4bb09bb06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:13.060910 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c676197-deff-4258-9ff9-56b4bb09bb06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:13.062283 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c676197-deff-4258-9ff9-56b4bb09bb06 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:13.074703 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 169/346] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:13.074703 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c676197-deff-4258-9ff9-56b4bb09bb06 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:13.088518 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41103e4-19ae-4ee2-8580-a8ffcf8f0aea None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:13.123682 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a14de-665c-4ca5-9ab0-05c194dae901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:13.124314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a14de-665c-4ca5-9ab0-05c194dae901 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:13.124713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a14de-665c-4ca5-9ab0-05c194dae901 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:13.125021 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a14de-665c-4ca5-9ab0-05c194dae901 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:13.137063 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38a14de-665c-4ca5-9ab0-05c194dae901 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:13.138763 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38a14de-665c-4ca5-9ab0-05c194dae901 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:13.155185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-e38a14de-665c-4ca5-9ab0-05c194dae901 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:13.155185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:13.155185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:13.155185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:13.155185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:13.155185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:13.155185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:13.155185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:13.155185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:13.155185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:13.155185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:13.155185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:13.155185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:13.155185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:13.155185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:13.155185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:13.156113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:13.156113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:13.156113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:13.156113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:13.156113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:13.156113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:13.156113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:13.156113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:13.156113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:13.156113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:13.156113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:13.156113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:13.156113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:13.156113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:13.156113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:13.156113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:13.156113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:13.156959 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:13.156959 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:13.156959 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:13.156959 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 178/347] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:13.157909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bc511-3872-40e8-a947-4b34d0721171 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:13.158190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bc511-3872-40e8-a947-4b34d0721171 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:13.158456 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bc511-3872-40e8-a947-4b34d0721171 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:13.159816 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 179/348] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:13.164676 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bc511-3872-40e8-a947-4b34d0721171 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:13.192990 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c676197-deff-4258-9ff9-56b4bb09bb06 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:13.228047 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137519f-eb0c-433c-a620-1add84542041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:13.228635 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137519f-eb0c-433c-a620-1add84542041 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:13.229122 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137519f-eb0c-433c-a620-1add84542041 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:13.229392 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137519f-eb0c-433c-a620-1add84542041 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:13.230235 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137519f-eb0c-433c-a620-1add84542041 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:13.231868 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137519f-eb0c-433c-a620-1add84542041 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:13.255427 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 170/349] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 20 13:23:13 2022] POST /identity/v3/users => generated 246 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:23:13.256612 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137519f-eb0c-433c-a620-1add84542041 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:13.290523 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bc511-3872-40e8-a947-4b34d0721171 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:13.294769 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:13.329939 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:13.350385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1032ce-e065-44b9-a6f1-bdba5e1f32d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:13.351937 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1032ce-e065-44b9-a6f1-bdba5e1f32d3 None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:13.352698 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1032ce-e065-44b9-a6f1-bdba5e1f32d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:13.353219 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1032ce-e065-44b9-a6f1-bdba5e1f32d3 None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:13.354372 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1032ce-e065-44b9-a6f1-bdba5e1f32d3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:13.356949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1032ce-e065-44b9-a6f1-bdba5e1f32d3 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:13.389470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 180/350] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 20 13:23:13 2022] POST /identity/v3/users => generated 248 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:23:13.391144 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b0572-362d-4368-a161-ac450f5692c1 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:13.391438 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b0572-362d-4368-a161-ac450f5692c1 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:13.391716 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b0572-362d-4368-a161-ac450f5692c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:13.405976 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137519f-eb0c-433c-a620-1add84542041 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:13.427371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-3a3b0572-362d-4368-a161-ac450f5692c1 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:13.428622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-3a3b0572-362d-4368-a161-ac450f5692c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:13.453288 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f3d8e-6372-4dc4-b7ac-61e4398d2c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:13.455794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f3d8e-6372-4dc4-b7ac-61e4398d2c2d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:13.456358 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f3d8e-6372-4dc4-b7ac-61e4398d2c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:13.456834 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f3d8e-6372-4dc4-b7ac-61e4398d2c2d None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:13.458556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f3d8e-6372-4dc4-b7ac-61e4398d2c2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:13.459765 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f3d8e-6372-4dc4-b7ac-61e4398d2c2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:13.462349 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f3d8e-6372-4dc4-b7ac-61e4398d2c2d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:13.482051 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 171/351] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:13.489007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f3d8e-6372-4dc4-b7ac-61e4398d2c2d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:13.551405 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 181/352] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:13.610856 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a40966-c24c-4678-85b7-d35e8bedf803 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:13.611484 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a40966-c24c-4678-85b7-d35e8bedf803 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:13.611951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a40966-c24c-4678-85b7-d35e8bedf803 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:13.613888 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 182/353] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:23:13.620754 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57856840-0310-4420-bea0-6cc7d22b2c45 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:13.621396 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57856840-0310-4420-bea0-6cc7d22b2c45 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:13.621858 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57856840-0310-4420-bea0-6cc7d22b2c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:13.624663 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f3d8e-6372-4dc4-b7ac-61e4398d2c2d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:13.627112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:13.655551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-57856840-0310-4420-bea0-6cc7d22b2c45 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:13.658019 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-57856840-0310-4420-bea0-6cc7d22b2c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:13.674810 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaceebdd-040d-4814-b445-6ae29ad79f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:13.676228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaceebdd-040d-4814-b445-6ae29ad79f93 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:13.676228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaceebdd-040d-4814-b445-6ae29ad79f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:13.676228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaceebdd-040d-4814-b445-6ae29ad79f93 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:13.676779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaceebdd-040d-4814-b445-6ae29ad79f93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:13.678925 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaceebdd-040d-4814-b445-6ae29ad79f93 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:13.709064 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 172/354] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 20 13:23:13 2022] POST /identity/v3/users => generated 248 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:23:13.748816 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 183/355] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:13.881579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaceebdd-040d-4814-b445-6ae29ad79f93 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:13.988071 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaceebdd-040d-4814-b445-6ae29ad79f93 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:14.023756 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e68c3f-0fff-4fa4-8f99-41584ce798bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:14.024499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e68c3f-0fff-4fa4-8f99-41584ce798bc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:14.024657 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e68c3f-0fff-4fa4-8f99-41584ce798bc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:14.024951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e68c3f-0fff-4fa4-8f99-41584ce798bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:14.035814 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e68c3f-0fff-4fa4-8f99-41584ce798bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:14.038058 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e68c3f-0fff-4fa4-8f99-41584ce798bc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:14.052928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-15e68c3f-0fff-4fa4-8f99-41584ce798bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:14.052928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:14.052928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:14.052928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:14.052928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:14.052928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:14.052928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:14.052928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:14.052928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:14.052928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:14.052928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:14.052928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:14.052928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:14.052928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:14.052928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:14.052928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:14.053854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:14.053854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:14.053854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:14.053854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:14.053854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:14.053854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:14.053854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:14.053854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:14.053854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:14.053854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:14.053854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:14.053854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:14.053854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:14.053854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:14.053854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:14.053854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:14.053854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:14.054696 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:14.054696 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:14.054696 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:14.056773 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 173/356] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:14.060528 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57856840-0310-4420-bea0-6cc7d22b2c45 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:14.072315 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e68c3f-0fff-4fa4-8f99-41584ce798bc None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:14.089697 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:14.095137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:14.198587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e68c3f-0fff-4fa4-8f99-41584ce798bc None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:14.236757 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1a229-8559-4be9-9665-69c2cbf3556a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:14.237845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1a229-8559-4be9-9665-69c2cbf3556a None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:14.238269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1a229-8559-4be9-9665-69c2cbf3556a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:14.238553 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1a229-8559-4be9-9665-69c2cbf3556a None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:14.239864 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe1a229-8559-4be9-9665-69c2cbf3556a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:14.241569 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe1a229-8559-4be9-9665-69c2cbf3556a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:14.249580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57856840-0310-4420-bea0-6cc7d22b2c45 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:14.257721 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.sql.core [None req-ebe1a229-8559-4be9-9665-69c2cbf3556a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 20 13:23:14.257721 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 20 13:23:14.257721 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [parameters: {'id': '8501b95b83224d3392521dd598dd375b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 20 13:23:14.257721 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81475) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 20 13:23:14.266190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-ebe1a229-8559-4be9-9665-69c2cbf3556a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 20 13:23:14.266190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:14.266190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 13:23:14.266190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 13:23:14.266190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 13:23:14.266190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 13:23:14.266190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 13:23:14.266190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 13:23:14.266190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 13:23:14.266190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application conn.query(q) Oct 20 13:23:14.266190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 13:23:14.266190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 13:23:14.266190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 13:23:14.266190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result.read() Oct 20 13:23:14.266190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 13:23:14.267029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 13:23:14.267029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 13:23:14.267029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 13:23:14.267029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 13:23:14.267029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 13:23:14.267029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 13:23:14.267029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 13:23:14.267029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 20 13:23:14.267029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:14.267029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 20 13:23:14.267029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:14.267029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:14.267029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 20 13:23:14.267029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 20 13:23:14.267029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 20 13:23:14.267029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return ref.to_dict() Oct 20 13:23:14.267029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 13:23:14.267029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application next(self.gen) Oct 20 13:23:14.267923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 20 13:23:14.267923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application yield resource Oct 20 13:23:14.267923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 13:23:14.267923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application next(self.gen) Oct 20 13:23:14.267923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 20 13:23:14.267923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.session.rollback() Oct 20 13:23:14.267923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 20 13:23:14.267923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.force_reraise() Oct 20 13:23:14.267923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 20 13:23:14.267923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise self.value Oct 20 13:23:14.267923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 20 13:23:14.267923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 20 13:23:14.267923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 20 13:23:14.267923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application session.commit() Oct 20 13:23:14.267923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 20 13:23:14.267923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 20 13:23:14.267923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 20 13:23:14.268777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._prepare_impl() Oct 20 13:23:14.268777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 20 13:23:14.268777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.session.flush() Oct 20 13:23:14.268777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 20 13:23:14.268777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._flush(objects) Oct 20 13:23:14.268777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 20 13:23:14.268777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 20 13:23:14.268777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 20 13:23:14.268777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application compat.raise_( Oct 20 13:23:14.268777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 13:23:14.268777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception Oct 20 13:23:14.268777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 20 13:23:14.268777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application flush_context.execute() Oct 20 13:23:14.268777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 20 13:23:14.268777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rec.execute(self) Oct 20 13:23:14.268777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 20 13:23:14.268777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 20 13:23:14.269639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 20 13:23:14.269639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 20 13:23:14.269639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 20 13:23:14.269639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 20 13:23:14.269639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 20 13:23:14.269639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 20 13:23:14.269639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 20 13:23:14.269639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 20 13:23:14.269639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 20 13:23:14.269639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 20 13:23:14.269639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 20 13:23:14.269639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 20 13:23:14.269639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 20 13:23:14.269639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 20 13:23:14.269639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 13:23:14.269639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception Oct 20 13:23:14.272227 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 13:23:14.272227 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 13:23:14.272227 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 13:23:14.272227 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 13:23:14.272227 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 13:23:14.272227 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 13:23:14.272227 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 13:23:14.272227 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application conn.query(q) Oct 20 13:23:14.272227 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 13:23:14.272227 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 13:23:14.272227 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 13:23:14.272227 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application result.read() Oct 20 13:23:14.272227 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 13:23:14.272227 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 13:23:14.272227 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 13:23:14.272227 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 13:23:14.277961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 13:23:14.277961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 13:23:14.277961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 13:23:14.277961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 13:23:14.277961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 20 13:23:14.277961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 20 13:23:14.277961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [parameters: {'id': '8501b95b83224d3392521dd598dd375b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 20 13:23:14.277961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 20 13:23:14.277961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:14.277961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:23:14.277961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:14.277961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:14.277961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:14.277961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:14.277961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:14.277961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:14.279896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:14.279896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:14.279896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:14.279896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:14.279896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:14.279896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:14.279896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 20 13:23:14.279896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 20 13:23:14.279896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:14.279896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:14.279896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 20 13:23:14.279896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 20 13:23:14.279896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 20 13:23:14.279896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 20 13:23:14.279896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 20 13:23:14.279896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:14.281870 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 174/357] 10.222.0.23 () {60 vars in 1131 bytes} [Thu Oct 20 13:23:14 2022] POST /identity/v3/roles => generated 144 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:23:14.281870 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1a229-8559-4be9-9665-69c2cbf3556a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:14.313187 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db5c3b-7d6b-4afc-a2fe-6572f6f6b3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:14.313805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db5c3b-7d6b-4afc-a2fe-6572f6f6b3ec None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:14.314131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db5c3b-7d6b-4afc-a2fe-6572f6f6b3ec None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:14.314413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db5c3b-7d6b-4afc-a2fe-6572f6f6b3ec None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:14.315066 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db5c3b-7d6b-4afc-a2fe-6572f6f6b3ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:14.315368 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db5c3b-7d6b-4afc-a2fe-6572f6f6b3ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:14.316820 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db5c3b-7d6b-4afc-a2fe-6572f6f6b3ec None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:14.341249 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 184/358] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:14.346392 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db5c3b-7d6b-4afc-a2fe-6572f6f6b3ec None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:14.435384 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:14.470677 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1a229-8559-4be9-9665-69c2cbf3556a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:14.489321 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:14.501653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db5c3b-7d6b-4afc-a2fe-6572f6f6b3ec None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:14.540973 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92cf364-450c-4a74-9b7f-7fef4995c662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:14.541618 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92cf364-450c-4a74-9b7f-7fef4995c662 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:14.541898 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92cf364-450c-4a74-9b7f-7fef4995c662 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:14.542173 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92cf364-450c-4a74-9b7f-7fef4995c662 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:14.563376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c187c2-261f-4fe9-a3ae-e44ed1a379fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:14.563995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c187c2-261f-4fe9-a3ae-e44ed1a379fa None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:14.564276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c187c2-261f-4fe9-a3ae-e44ed1a379fa None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:14.564553 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c187c2-261f-4fe9-a3ae-e44ed1a379fa None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:14.565235 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c187c2-261f-4fe9-a3ae-e44ed1a379fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:14.566674 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c187c2-261f-4fe9-a3ae-e44ed1a379fa None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:14.569891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92cf364-450c-4a74-9b7f-7fef4995c662 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:14.579588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-b92cf364-450c-4a74-9b7f-7fef4995c662 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 13:23:14.579588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:14.579588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:14.579588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:14.579588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:14.579588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:14.579588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:14.579588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:14.579588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:14.579588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:14.579588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:14.579588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:14.579588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:14.579588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:14.579588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:14.579588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:14.581032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:14.581032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:14.581032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:14.581032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:14.581032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:14.581032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:14.581032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:14.581032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:14.581032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:14.581032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:14.581032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:14.581032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:14.581032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:14.581032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:14.581032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:14.581032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:14.581850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:14.581850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:14.581850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:14.581850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:14.581850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:14.581850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:14.581850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:14.581850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:14.581850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:14.581850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:14.581850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 20 13:23:14.581850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:14.581850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 175/359] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 20 13:23:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:23:14.587695 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92cf364-450c-4a74-9b7f-7fef4995c662 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:14.601619 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 185/360] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 20 13:23:14 2022] POST /identity/v3/users => generated 249 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:23:14.742998 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92cf364-450c-4a74-9b7f-7fef4995c662 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:14.794246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3947a81-2e91-4a7e-80bb-c1792ccb844c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:14.794875 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3947a81-2e91-4a7e-80bb-c1792ccb844c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:14.795156 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3947a81-2e91-4a7e-80bb-c1792ccb844c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:14.795428 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3947a81-2e91-4a7e-80bb-c1792ccb844c None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:14.796109 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3947a81-2e91-4a7e-80bb-c1792ccb844c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:14.796411 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3947a81-2e91-4a7e-80bb-c1792ccb844c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:14.797936 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3947a81-2e91-4a7e-80bb-c1792ccb844c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:14.804888 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:14.813393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 176/361] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:15.950408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ab57e-a813-4f13-a84e-4558f1435b87 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:15.950946 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ab57e-a813-4f13-a84e-4558f1435b87 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:15.950946 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ab57e-a813-4f13-a84e-4558f1435b87 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:15.952574 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 186/362] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:15.959219 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2d082-9c90-44e7-b58c-865728180203 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:15.959750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2d082-9c90-44e7-b58c-865728180203 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:15.959824 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2d082-9c90-44e7-b58c-865728180203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:15.986003 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-17c2d082-9c90-44e7-b58c-865728180203 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:15.987246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-17c2d082-9c90-44e7-b58c-865728180203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:16.081068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd6090-544d-4f1d-945c-cbf81dad04d6 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:16.081707 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd6090-544d-4f1d-945c-cbf81dad04d6 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:16.082172 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd6090-544d-4f1d-945c-cbf81dad04d6 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:16.084465 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 187/363] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:16.086452 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 177/364] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:16.090079 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab861b67-59fc-409d-b798-cdbe4a04b696 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:16.090570 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab861b67-59fc-409d-b798-cdbe4a04b696 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:16.091032 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab861b67-59fc-409d-b798-cdbe4a04b696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:16.133860 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-ab861b67-59fc-409d-b798-cdbe4a04b696 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:16.135391 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-ab861b67-59fc-409d-b798-cdbe4a04b696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:16.264122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 188/365] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 13:23:16.353699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c2d082-9c90-44e7-b58c-865728180203 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:16.498714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c2d082-9c90-44e7-b58c-865728180203 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:16.537400 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:16.546347 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f38db-d7bf-4116-9254-3d410c5d38c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:16.547011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f38db-d7bf-4116-9254-3d410c5d38c6 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:16.547296 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f38db-d7bf-4116-9254-3d410c5d38c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:16.547575 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f38db-d7bf-4116-9254-3d410c5d38c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:16.559812 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f38db-d7bf-4116-9254-3d410c5d38c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:16.564779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-8e3f38db-d7bf-4116-9254-3d410c5d38c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:23:16.564779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:16.564779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:16.564779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:16.564779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:16.564779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:16.564779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:16.564779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:16.564779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:16.564779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:16.564779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:16.564779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:16.564779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:16.564779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:16.564779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:16.564779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:16.565650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:16.565650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:16.565650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:16.565650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:16.565650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:16.565650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:16.565650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:16.565650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:16.565650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:16.565650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:16.565650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:16.565650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:16.565650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:16.565650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:16.565650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:16.565650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:16.566420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:16.566420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:16.566420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:16.566420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:16.566420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:16.566420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:16.566420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:16.566420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:16.566420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:16.566420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:16.566420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:23:16.566420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:16.566420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 178/366] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 20 13:23:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:16.572881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab861b67-59fc-409d-b798-cdbe4a04b696 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:16.664524 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:16.715605 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab861b67-59fc-409d-b798-cdbe4a04b696 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:16.737365 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b605f7-6c0f-472a-86d8-7a0a311c8496 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:16.737964 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b605f7-6c0f-472a-86d8-7a0a311c8496 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:16.738436 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b605f7-6c0f-472a-86d8-7a0a311c8496 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:16.744999 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 179/367] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:23:16.746683 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a160f-50d0-414f-b3d5-e3894511c29d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:16.747142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a160f-50d0-414f-b3d5-e3894511c29d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:16.747607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a160f-50d0-414f-b3d5-e3894511c29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:16.762955 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08dd35f-a376-42b2-b85e-932faf4c9dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:16.763783 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08dd35f-a376-42b2-b85e-932faf4c9dce None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:16.764223 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08dd35f-a376-42b2-b85e-932faf4c9dce None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:16.764651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08dd35f-a376-42b2-b85e-932faf4c9dce None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:16.765484 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08dd35f-a376-42b2-b85e-932faf4c9dce None admin] RBAC: Adding query filter params (name=service) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:16.765913 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08dd35f-a376-42b2-b85e-932faf4c9dce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:16.767401 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08dd35f-a376-42b2-b85e-932faf4c9dce None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:16.778061 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-371a160f-50d0-414f-b3d5-e3894511c29d None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:16.779462 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-371a160f-50d0-414f-b3d5-e3894511c29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:16.784743 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 189/368] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 20 13:23:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:16.785665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08dd35f-a376-42b2-b85e-932faf4c9dce None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:16.850182 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 180/369] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 13:23:16.851318 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a160f-50d0-414f-b3d5-e3894511c29d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:16.912821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08dd35f-a376-42b2-b85e-932faf4c9dce None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:16.940779 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:16.950726 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c497e-e44c-4e4c-9686-f1fd7da5b347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:16.952039 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c497e-e44c-4e4c-9686-f1fd7da5b347 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:16.952502 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c497e-e44c-4e4c-9686-f1fd7da5b347 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:16.952954 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c497e-e44c-4e4c-9686-f1fd7da5b347 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:16.965056 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276c497e-e44c-4e4c-9686-f1fd7da5b347 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:16.970721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-276c497e-e44c-4e4c-9686-f1fd7da5b347 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:23:16.970721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:16.970721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:16.970721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:16.970721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:16.970721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:16.970721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:16.970721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:16.970721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:16.970721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:16.970721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:16.970721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:16.970721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:16.970721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:16.970721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:16.970721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:16.971798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:16.971798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:16.971798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:16.971798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:16.971798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:16.971798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:16.971798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:16.971798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:16.971798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:16.971798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:16.971798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:16.971798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:16.971798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:16.971798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:16.971798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:16.971798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:16.972794 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:16.972794 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:16.972794 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:16.972794 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:16.972794 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:16.972794 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:16.972794 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:16.972794 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:16.972794 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:16.972794 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:16.972794 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:23:16.972794 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:16.974794 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 190/370] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 20 13:23:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:16.980584 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c497e-e44c-4e4c-9686-f1fd7da5b347 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:17.007352 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a160f-50d0-414f-b3d5-e3894511c29d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:17.039565 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:17.078479 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffffc5-1406-4128-ae40-a5e2a1c87e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:17.079124 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffffc5-1406-4128-ae40-a5e2a1c87e5c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:17.079406 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffffc5-1406-4128-ae40-a5e2a1c87e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:17.079692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffffc5-1406-4128-ae40-a5e2a1c87e5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:17.090672 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c497e-e44c-4e4c-9686-f1fd7da5b347 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:17.095303 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ffffc5-1406-4128-ae40-a5e2a1c87e5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:17.096761 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ffffc5-1406-4128-ae40-a5e2a1c87e5c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:17.111953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-16ffffc5-1406-4128-ae40-a5e2a1c87e5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:17.111953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:17.111953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:17.111953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:17.111953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:17.111953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:17.111953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:17.111953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:17.111953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:17.111953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:17.111953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:17.111953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:17.111953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:17.111953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:17.111953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:17.111953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:17.112883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:17.112883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:17.112883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:17.112883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:17.112883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:17.112883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:17.112883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:17.112883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:17.112883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:17.112883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:17.112883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:17.112883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:17.112883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:17.112883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:17.112883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:17.112883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:17.112883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:17.113742 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:17.113742 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:17.113742 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:17.113742 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 181/371] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:17.119109 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffffc5-1406-4128-ae40-a5e2a1c87e5c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:17.130230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc44ce-f5e5-430e-a327-b97158e96f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:17.131162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc44ce-f5e5-430e-a327-b97158e96f0e None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:17.131639 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc44ce-f5e5-430e-a327-b97158e96f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:17.132124 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc44ce-f5e5-430e-a327-b97158e96f0e None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:17.133012 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc44ce-f5e5-430e-a327-b97158e96f0e None admin] RBAC: Adding query filter params (name=service) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:17.133512 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc44ce-f5e5-430e-a327-b97158e96f0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:17.135100 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc44ce-f5e5-430e-a327-b97158e96f0e None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:17.144344 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 191/372] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 20 13:23:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:17.150557 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc44ce-f5e5-430e-a327-b97158e96f0e None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:17.203112 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:17.222568 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffffc5-1406-4128-ae40-a5e2a1c87e5c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:17.257993 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0824a8-4709-41c8-9c39-cdecfe76a336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:17.258598 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0824a8-4709-41c8-9c39-cdecfe76a336 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:17.258874 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0824a8-4709-41c8-9c39-cdecfe76a336 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:17.259143 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0824a8-4709-41c8-9c39-cdecfe76a336 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:17.259593 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc44ce-f5e5-430e-a327-b97158e96f0e None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:17.259783 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0824a8-4709-41c8-9c39-cdecfe76a336 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:17.260083 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0824a8-4709-41c8-9c39-cdecfe76a336 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:17.261477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0824a8-4709-41c8-9c39-cdecfe76a336 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:17.278490 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 182/373] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:23:17.279815 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0824a8-4709-41c8-9c39-cdecfe76a336 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:17.303945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ffba1-c646-4013-bc84-24356dd4bf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:17.309257 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ffba1-c646-4013-bc84-24356dd4bf15 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:17.309881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ffba1-c646-4013-bc84-24356dd4bf15 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:17.310350 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ffba1-c646-4013-bc84-24356dd4bf15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:17.314615 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:17.333649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ffba1-c646-4013-bc84-24356dd4bf15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:17.335127 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ffba1-c646-4013-bc84-24356dd4bf15 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:17.365325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-f88ffba1-c646-4013-bc84-24356dd4bf15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:17.365325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:17.365325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:17.365325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:17.365325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:17.365325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:17.365325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:17.365325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:17.365325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:17.365325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:17.365325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:17.365325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:17.365325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:17.365325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:17.365325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:17.365325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:17.366212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:17.366212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:17.366212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:17.366212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:17.366212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:17.366212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:17.366212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:17.366212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:17.366212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:17.366212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:17.366212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:17.366212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:17.366212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:17.366212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:17.366212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:17.366212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:17.366212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:17.367284 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:17.367284 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:17.367284 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:17.367284 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 192/374] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:17.372536 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ffba1-c646-4013-bc84-24356dd4bf15 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:17.391315 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0824a8-4709-41c8-9c39-cdecfe76a336 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:17.427522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef203c96-fbb3-4703-9871-81396217f39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:17.428419 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef203c96-fbb3-4703-9871-81396217f39a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:17.428835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef203c96-fbb3-4703-9871-81396217f39a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:17.429137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef203c96-fbb3-4703-9871-81396217f39a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:17.436264 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef203c96-fbb3-4703-9871-81396217f39a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:17.437723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef203c96-fbb3-4703-9871-81396217f39a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:17.451112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-ef203c96-fbb3-4703-9871-81396217f39a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 20 13:23:17.451112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:17.451112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:17.451112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:17.451112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:17.451112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:17.451112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:17.451112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:17.451112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:17.451112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:17.451112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:17.451112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:17.451112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:23:17.451112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:23:17.451112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:23:17.451112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:23:17.452007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:17.452007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:17.452007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:23:17.452007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:17.452007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:23:17.452007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:17.452007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:17.452007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:17.452007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:17.452007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:17.452007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:17.452007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:17.452007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:17.452007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:17.452007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:17.452007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:17.452007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:17.452007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:17.452944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:17.452944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:17.452944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:23:17.452944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:23:17.452944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:23:17.452944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:23:17.452944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:23:17.452944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:23:17.452944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 20 13:23:17.452944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:17.456695 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 183/375] 10.222.0.23 () {56 vars in 1093 bytes} [Thu Oct 20 13:23:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:17.459102 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad0216-09da-400d-a33e-cb87fa9bc0bd None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:17.459102 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad0216-09da-400d-a33e-cb87fa9bc0bd None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:17.459102 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad0216-09da-400d-a33e-cb87fa9bc0bd None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:17.460344 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 184/376] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:17.461178 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad0216-09da-400d-a33e-cb87fa9bc0bd None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:17.494797 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ffba1-c646-4013-bc84-24356dd4bf15 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:17.496950 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:17.539240 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a0344-e37e-4f58-9320-8dd165b122f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:17.539852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a0344-e37e-4f58-9320-8dd165b122f4 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:17.540132 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a0344-e37e-4f58-9320-8dd165b122f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:17.540413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a0344-e37e-4f58-9320-8dd165b122f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:17.541084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a0344-e37e-4f58-9320-8dd165b122f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:17.541389 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a0344-e37e-4f58-9320-8dd165b122f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:17.542769 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a0344-e37e-4f58-9320-8dd165b122f4 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:17.554496 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 193/377] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:23:17.559639 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a0344-e37e-4f58-9320-8dd165b122f4 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:17.574286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:17.588012 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad0216-09da-400d-a33e-cb87fa9bc0bd None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:17.623133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761e20e-131f-4e53-ad08-bf4b0c9a7f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:17.623682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761e20e-131f-4e53-ad08-bf4b0c9a7f45 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:17.623959 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761e20e-131f-4e53-ad08-bf4b0c9a7f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:17.624230 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761e20e-131f-4e53-ad08-bf4b0c9a7f45 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:17.634614 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761e20e-131f-4e53-ad08-bf4b0c9a7f45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:17.640087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-5761e20e-131f-4e53-ad08-bf4b0c9a7f45 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:23:17.640087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:17.640087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:17.640087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:17.640087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:17.640087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:17.640087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:17.640087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:17.640087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:17.640087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:17.640087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:17.640087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:17.640087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:17.640087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:17.640087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:17.640087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:17.641010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:17.641010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:17.641010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:17.641010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:17.641010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:17.641010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:17.641010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:17.641010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:17.641010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:17.641010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:17.641010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:17.641010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:17.641010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:17.641010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:17.641010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:17.641010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:17.641801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:17.641801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:17.641801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:17.641801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:17.641801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:17.641801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:17.641801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:17.641801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:17.641801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:17.641801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:17.641801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:23:17.641801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:17.641801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 185/378] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 20 13:23:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:17.642463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73013c0f-e1c9-4fc1-a893-47c5be30eea9 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:17.642624 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73013c0f-e1c9-4fc1-a893-47c5be30eea9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:17.642894 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73013c0f-e1c9-4fc1-a893-47c5be30eea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:17.659941 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-73013c0f-e1c9-4fc1-a893-47c5be30eea9 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:17.660455 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a0344-e37e-4f58-9320-8dd165b122f4 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:17.661335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-73013c0f-e1c9-4fc1-a893-47c5be30eea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:17.695048 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575dc4fc-48b2-4819-80f9-c67aabc4ba4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:17.695657 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575dc4fc-48b2-4819-80f9-c67aabc4ba4b None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:17.695933 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575dc4fc-48b2-4819-80f9-c67aabc4ba4b None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:17.696205 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575dc4fc-48b2-4819-80f9-c67aabc4ba4b None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:17.696958 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575dc4fc-48b2-4819-80f9-c67aabc4ba4b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:17.697265 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575dc4fc-48b2-4819-80f9-c67aabc4ba4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:17.698699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575dc4fc-48b2-4819-80f9-c67aabc4ba4b None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:17.728078 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 186/379] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:17.729273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73013c0f-e1c9-4fc1-a893-47c5be30eea9 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:17.744580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 194/380] 10.222.0.23 () {56 vars in 1133 bytes} [Thu Oct 20 13:23:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:17.745867 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65461187-2ed0-4c16-a182-f4a88b962424 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:17.746155 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65461187-2ed0-4c16-a182-f4a88b962424 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:17.746435 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65461187-2ed0-4c16-a182-f4a88b962424 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:17.748682 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 195/381] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:17.748682 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65461187-2ed0-4c16-a182-f4a88b962424 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:17.773178 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:17.822852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:17.845887 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73013c0f-e1c9-4fc1-a893-47c5be30eea9 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:17.858727 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65461187-2ed0-4c16-a182-f4a88b962424 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:17.882953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951a131-99f1-4b83-ae48-17d3704869bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:17.884014 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951a131-99f1-4b83-ae48-17d3704869bc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:17.884339 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951a131-99f1-4b83-ae48-17d3704869bc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:17.884622 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951a131-99f1-4b83-ae48-17d3704869bc None admin] PATH_INFO: `/v3/users/glance` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:17.892588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951a131-99f1-4b83-ae48-17d3704869bc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:17.894072 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951a131-99f1-4b83-ae48-17d3704869bc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:17.894270 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca358f-709c-41af-9013-72a5f509a7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:17.894893 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca358f-709c-41af-9013-72a5f509a7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:17.895148 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca358f-709c-41af-9013-72a5f509a7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:17.896730 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca358f-709c-41af-9013-72a5f509a7a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:17.897442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca358f-709c-41af-9013-72a5f509a7a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:17.897751 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca358f-709c-41af-9013-72a5f509a7a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:17.899137 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca358f-709c-41af-9013-72a5f509a7a8 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:17.906910 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 196/382] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 20 13:23:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:17.907687 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca358f-709c-41af-9013-72a5f509a7a8 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:17.908042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-6951a131-99f1-4b83-ae48-17d3704869bc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 20 13:23:17.908042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:17.908042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:17.908042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:17.908042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:17.908042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:17.908042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:17.908042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:17.908042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:17.908042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:17.908042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:17.908042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:17.908042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:23:17.908042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:23:17.908042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:23:17.908042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:23:17.909010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:17.909010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:17.909010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:23:17.909010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:17.909010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:23:17.909010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:17.909010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:17.909010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:17.909010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:17.909010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:17.909010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:17.909010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:17.909010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:17.909010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:17.909010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:17.909010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:17.909010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:17.909010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:17.909902 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:17.909902 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:17.909902 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:23:17.909902 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:23:17.909902 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:23:17.909902 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:23:17.909902 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:23:17.909902 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:23:17.909902 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 20 13:23:17.909902 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:17.913023 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 187/383] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:23:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:17.914550 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11185e3-a083-42c6-82e8-1905a72a971a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:17.914835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11185e3-a083-42c6-82e8-1905a72a971a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:17.915106 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11185e3-a083-42c6-82e8-1905a72a971a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:17.932588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b11185e3-a083-42c6-82e8-1905a72a971a None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:17.934097 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b11185e3-a083-42c6-82e8-1905a72a971a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:17.999773 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:18.004359 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 188/384] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:18.006007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11185e3-a083-42c6-82e8-1905a72a971a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:18.022162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca358f-709c-41af-9013-72a5f509a7a8 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:18.058821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddfa29-aacd-417a-ab1c-c64751e954c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:18.059453 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddfa29-aacd-417a-ab1c-c64751e954c8 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:18.059737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddfa29-aacd-417a-ab1c-c64751e954c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:18.060012 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ddfa29-aacd-417a-ab1c-c64751e954c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:18.071359 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ddfa29-aacd-417a-ab1c-c64751e954c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:18.073064 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ddfa29-aacd-417a-ab1c-c64751e954c8 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:18.080037 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:18.087635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-30ddfa29-aacd-417a-ab1c-c64751e954c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:18.087635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:18.087635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:18.087635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:18.087635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:18.087635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:18.087635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:18.087635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:18.087635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:18.087635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:18.087635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:18.087635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:18.087635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:18.087635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:18.087635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:18.087635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:18.088637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:18.088637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:18.088637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:18.088637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:18.088637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:18.088637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:18.088637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:18.088637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:18.088637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:18.088637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:18.088637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:18.088637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:18.088637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:18.088637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:18.088637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:18.088637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:18.088637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:18.089812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:18.089812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:18.089812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:18.089812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 197/385] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:18.090033 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddfa29-aacd-417a-ab1c-c64751e954c8 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:18.119092 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11185e3-a083-42c6-82e8-1905a72a971a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:18.155629 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6bdd4-b660-4699-9766-200e59cd0d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:18.156082 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6bdd4-b660-4699-9766-200e59cd0d1a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:18.156365 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6bdd4-b660-4699-9766-200e59cd0d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:18.156897 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6bdd4-b660-4699-9766-200e59cd0d1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:18.167398 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba6bdd4-b660-4699-9766-200e59cd0d1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:18.168858 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba6bdd4-b660-4699-9766-200e59cd0d1a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:18.183918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-8ba6bdd4-b660-4699-9766-200e59cd0d1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:18.183918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:18.183918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:18.183918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:18.183918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:18.183918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:18.183918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:18.183918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:18.183918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:18.183918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:18.183918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:18.183918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:18.183918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:18.183918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:18.183918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:18.183918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:18.184809 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:18.184809 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:18.184809 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:18.184809 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:18.184809 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:18.184809 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:18.184809 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:18.184809 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:18.184809 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:18.184809 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:18.184809 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:18.184809 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:18.184809 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:18.184809 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:18.184809 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:18.184809 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:18.184809 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:18.185690 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:18.185690 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:18.185690 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:18.185690 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 189/386] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:18.185897 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6bdd4-b660-4699-9766-200e59cd0d1a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:18.202809 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ddfa29-aacd-417a-ab1c-c64751e954c8 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:18.239583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f346fbb-5d73-4e94-bd7f-9a527ea13089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:18.240205 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f346fbb-5d73-4e94-bd7f-9a527ea13089 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:18.240494 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f346fbb-5d73-4e94-bd7f-9a527ea13089 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:18.240770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f346fbb-5d73-4e94-bd7f-9a527ea13089 None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:18.241533 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f346fbb-5d73-4e94-bd7f-9a527ea13089 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:18.241842 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f346fbb-5d73-4e94-bd7f-9a527ea13089 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:18.243309 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f346fbb-5d73-4e94-bd7f-9a527ea13089 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:18.245243 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:18.265806 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 198/387] 10.222.0.23 () {56 vars in 1137 bytes} [Thu Oct 20 13:23:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:18.266223 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f346fbb-5d73-4e94-bd7f-9a527ea13089 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:18.300311 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6bdd4-b660-4699-9766-200e59cd0d1a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:18.323769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:18.335149 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f30f3c-49be-4632-89fa-9eb23ce43a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:18.335948 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f30f3c-49be-4632-89fa-9eb23ce43a52 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:18.336220 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f30f3c-49be-4632-89fa-9eb23ce43a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:18.336484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f30f3c-49be-4632-89fa-9eb23ce43a52 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:18.346474 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f30f3c-49be-4632-89fa-9eb23ce43a52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:18.352292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-06f30f3c-49be-4632-89fa-9eb23ce43a52 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:23:18.352292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:18.352292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:18.352292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:18.352292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:18.352292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:18.352292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:18.352292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:18.352292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:18.352292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:18.352292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:18.352292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:18.352292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:18.352292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:18.352292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:18.352292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:18.353252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:18.353252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:18.353252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:18.353252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:18.353252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:18.353252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:18.353252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:18.353252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:18.353252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:18.353252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:18.353252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:18.353252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:18.353252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:18.353252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:18.353252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:18.353252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:18.354038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:18.354038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:18.354038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:18.354038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:18.354038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:18.354038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:18.354038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:18.354038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:18.354038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:18.354038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:18.354038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:23:18.354038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:18.354038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 190/388] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 20 13:23:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:18.354680 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f30f3c-49be-4632-89fa-9eb23ce43a52 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:18.364978 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f346fbb-5d73-4e94-bd7f-9a527ea13089 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:18.398805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbff973-f53a-4de8-b222-b46741b41dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:18.399417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbff973-f53a-4de8-b222-b46741b41dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:18.399696 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbff973-f53a-4de8-b222-b46741b41dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:18.399967 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbff973-f53a-4de8-b222-b46741b41dc7 None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:18.400612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbff973-f53a-4de8-b222-b46741b41dc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:18.400953 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbff973-f53a-4de8-b222-b46741b41dc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:18.402338 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbff973-f53a-4de8-b222-b46741b41dc7 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:18.413361 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 199/389] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:18.414089 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbff973-f53a-4de8-b222-b46741b41dc7 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:18.453660 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f30f3c-49be-4632-89fa-9eb23ce43a52 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:18.465842 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:18.486170 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9c9f5-d0fe-442b-905e-05a2879ab080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:18.486768 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9c9f5-d0fe-442b-905e-05a2879ab080 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:18.487046 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9c9f5-d0fe-442b-905e-05a2879ab080 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:18.487327 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9c9f5-d0fe-442b-905e-05a2879ab080 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:18.487958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9c9f5-d0fe-442b-905e-05a2879ab080 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:18.488260 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9c9f5-d0fe-442b-905e-05a2879ab080 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:18.490357 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9c9f5-d0fe-442b-905e-05a2879ab080 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:18.501945 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 191/390] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:18.502753 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9c9f5-d0fe-442b-905e-05a2879ab080 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:18.518146 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbff973-f53a-4de8-b222-b46741b41dc7 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:18.545588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:18.550916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b124e815-b46e-4a69-bb87-67c658c14f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:18.552085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124e815-b46e-4a69-bb87-67c658c14f77 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:18.552378 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124e815-b46e-4a69-bb87-67c658c14f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:18.552641 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124e815-b46e-4a69-bb87-67c658c14f77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:18.563611 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b124e815-b46e-4a69-bb87-67c658c14f77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:18.565095 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b124e815-b46e-4a69-bb87-67c658c14f77 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:18.580081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-b124e815-b46e-4a69-bb87-67c658c14f77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:18.580081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:18.580081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:18.580081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:18.580081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:18.580081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:18.580081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:18.580081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:18.580081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:18.580081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:18.580081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:18.580081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:18.580081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:18.580081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:18.580081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:18.580081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:18.581206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:18.581206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:18.581206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:18.581206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:18.581206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:18.581206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:18.581206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:18.581206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:18.581206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:18.581206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:18.581206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:18.581206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:18.581206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:18.581206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:18.581206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:18.581206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:18.581206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:18.582133 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:18.582133 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:18.582133 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:18.582133 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 200/391] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:18.582133 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b124e815-b46e-4a69-bb87-67c658c14f77 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:18.603132 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9c9f5-d0fe-442b-905e-05a2879ab080 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:18.638740 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457ad16-0537-47e9-b293-24b5d5f3d5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:18.639369 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457ad16-0537-47e9-b293-24b5d5f3d5bb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:18.639644 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457ad16-0537-47e9-b293-24b5d5f3d5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:18.639920 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457ad16-0537-47e9-b293-24b5d5f3d5bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:18.650191 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5457ad16-0537-47e9-b293-24b5d5f3d5bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:18.651822 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5457ad16-0537-47e9-b293-24b5d5f3d5bb None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:18.666579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-5457ad16-0537-47e9-b293-24b5d5f3d5bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:18.666579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:18.666579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:18.666579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:18.666579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:18.666579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:18.666579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:18.666579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:18.666579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:18.666579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:18.666579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:18.666579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:18.666579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:18.666579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:18.666579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:18.666579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:18.667578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:18.667578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:18.667578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:18.667578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:18.667578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:18.667578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:18.667578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:18.667578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:18.667578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:18.667578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:18.667578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:18.667578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:18.667578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:18.667578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:18.667578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:18.667578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:18.667578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:18.669022 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:18.669022 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:18.669022 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:18.669022 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 192/392] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:18.669022 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457ad16-0537-47e9-b293-24b5d5f3d5bb None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:18.682126 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b124e815-b46e-4a69-bb87-67c658c14f77 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:18.684392 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:18.716393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc7aa3-12ee-47d8-9730-d46f5a9146a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:18.717031 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc7aa3-12ee-47d8-9730-d46f5a9146a9 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:18.717309 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc7aa3-12ee-47d8-9730-d46f5a9146a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:18.717575 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc7aa3-12ee-47d8-9730-d46f5a9146a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:18.718220 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc7aa3-12ee-47d8-9730-d46f5a9146a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:18.718517 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc7aa3-12ee-47d8-9730-d46f5a9146a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:18.719874 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc7aa3-12ee-47d8-9730-d46f5a9146a9 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:18.728385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 201/393] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 20 13:23:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:18.729198 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc7aa3-12ee-47d8-9730-d46f5a9146a9 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:18.766785 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:18.774342 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457ad16-0537-47e9-b293-24b5d5f3d5bb None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:18.808368 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e031c9-810b-40fa-9bba-5b28f4a2cfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:18.809020 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e031c9-810b-40fa-9bba-5b28f4a2cfab None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:18.809324 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e031c9-810b-40fa-9bba-5b28f4a2cfab None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:18.809575 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e031c9-810b-40fa-9bba-5b28f4a2cfab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:18.816014 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e031c9-810b-40fa-9bba-5b28f4a2cfab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:18.818010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e031c9-810b-40fa-9bba-5b28f4a2cfab None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:18.824747 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc7aa3-12ee-47d8-9730-d46f5a9146a9 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:18.829513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-c9e031c9-810b-40fa-9bba-5b28f4a2cfab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:18.829513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:18.829513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:18.829513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:18.829513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:18.829513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:18.829513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:18.829513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:18.829513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:18.829513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:18.829513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:18.829513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:18.829513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:23:18.829513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:23:18.829513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:23:18.830335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:23:18.830335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:18.830335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:18.830335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:18.830335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:18.830335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:18.830335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:18.830335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:18.830335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:18.830335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:18.830335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:18.830335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:18.830335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:18.830335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:18.830335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:18.830335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:18.830335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:18.831171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:23:18.831171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:23:18.831171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:23:18.831171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:23:18.831171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:23:18.831171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:23:18.831171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:18.831171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:18.831171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 193/394] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:18.831899 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e031c9-810b-40fa-9bba-5b28f4a2cfab None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:18.861959 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff41e2-2382-4b85-85ad-c7df962e2c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:18.862576 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff41e2-2382-4b85-85ad-c7df962e2c29 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:18.862865 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff41e2-2382-4b85-85ad-c7df962e2c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:18.863135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff41e2-2382-4b85-85ad-c7df962e2c29 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:18.871052 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff41e2-2382-4b85-85ad-c7df962e2c29 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:18.872504 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff41e2-2382-4b85-85ad-c7df962e2c29 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:18.885182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-6fff41e2-2382-4b85-85ad-c7df962e2c29 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 20 13:23:18.885182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:18.885182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:18.885182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:18.885182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:18.885182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:18.885182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:18.885182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:18.885182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:18.885182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:18.885182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:18.885182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:18.885182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:23:18.885182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:23:18.885182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:23:18.885182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:23:18.886084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:18.886084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:18.886084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:23:18.886084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:18.886084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:23:18.886084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:18.886084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:18.886084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:18.886084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:18.886084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:18.886084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:18.886084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:18.886084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:18.886084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:18.886084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:18.886084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:18.886084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:18.886084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:18.886981 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:18.886981 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:18.886981 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:23:18.886981 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:23:18.886981 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:23:18.886981 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:23:18.886981 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:23:18.886981 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:23:18.886981 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 20 13:23:18.886981 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:18.886981 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 202/395] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:23:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:18.887532 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff41e2-2382-4b85-85ad-c7df962e2c29 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:18.915013 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:18.933853 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e031c9-810b-40fa-9bba-5b28f4a2cfab None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:18.967166 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4a39d-3414-40c3-8c97-97d37eda7ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:18.967762 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4a39d-3414-40c3-8c97-97d37eda7ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:18.968039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4a39d-3414-40c3-8c97-97d37eda7ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:18.968312 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4a39d-3414-40c3-8c97-97d37eda7ab2 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:18.968983 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4a39d-3414-40c3-8c97-97d37eda7ab2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:18.969290 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4a39d-3414-40c3-8c97-97d37eda7ab2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:18.970684 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4a39d-3414-40c3-8c97-97d37eda7ab2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:18.982937 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 194/396] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:18.984018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4a39d-3414-40c3-8c97-97d37eda7ab2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:18.988038 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff41e2-2382-4b85-85ad-c7df962e2c29 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:18.999197 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:19.021742 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb76dc-a15e-4522-9b01-b2b99b584e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:19.022340 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb76dc-a15e-4522-9b01-b2b99b584e8d None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:19.022620 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb76dc-a15e-4522-9b01-b2b99b584e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:19.022893 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb76dc-a15e-4522-9b01-b2b99b584e8d None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:19.023525 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb76dc-a15e-4522-9b01-b2b99b584e8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:19.023826 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb76dc-a15e-4522-9b01-b2b99b584e8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:19.025221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb76dc-a15e-4522-9b01-b2b99b584e8d None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:19.037080 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 203/397] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:19.037865 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb76dc-a15e-4522-9b01-b2b99b584e8d None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:19.084308 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4a39d-3414-40c3-8c97-97d37eda7ab2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:19.118067 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e99c8-f20f-47fe-93c8-e1ae4d2e5ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:19.118696 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e99c8-f20f-47fe-93c8-e1ae4d2e5ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:19.118961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e99c8-f20f-47fe-93c8-e1ae4d2e5ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:19.119230 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e99c8-f20f-47fe-93c8-e1ae4d2e5ef5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:19.122155 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:19.129260 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e99c8-f20f-47fe-93c8-e1ae4d2e5ef5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:19.131307 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e99c8-f20f-47fe-93c8-e1ae4d2e5ef5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:19.137366 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb76dc-a15e-4522-9b01-b2b99b584e8d None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:19.146602 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-b82e99c8-f20f-47fe-93c8-e1ae4d2e5ef5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:19.146602 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:19.146602 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:19.146602 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:19.146602 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:19.146602 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:19.146602 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:19.146602 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:19.146602 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:19.146602 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:19.146602 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:19.146602 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:19.146602 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:19.146602 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:19.146602 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:19.146602 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:19.147640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:19.147640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:19.147640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:19.147640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:19.147640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:19.147640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:19.147640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:19.147640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:19.147640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:19.147640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:19.147640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:19.147640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:19.147640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:19.147640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:19.147640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:19.147640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:19.147640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:19.148474 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:19.148474 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:19.148474 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:19.148474 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 195/398] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:19.148474 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e99c8-f20f-47fe-93c8-e1ae4d2e5ef5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:19.172492 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d504e1-3393-4ed7-8b01-d99aa39f9cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:19.173124 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d504e1-3393-4ed7-8b01-d99aa39f9cea None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:19.173405 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d504e1-3393-4ed7-8b01-d99aa39f9cea None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:19.173676 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d504e1-3393-4ed7-8b01-d99aa39f9cea None admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:19.174363 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d504e1-3393-4ed7-8b01-d99aa39f9cea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:19.174670 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d504e1-3393-4ed7-8b01-d99aa39f9cea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:19.176127 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d504e1-3393-4ed7-8b01-d99aa39f9cea None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:19.194748 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 204/399] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 20 13:23:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:19.195496 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d504e1-3393-4ed7-8b01-d99aa39f9cea None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:19.218094 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:19.252804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e99c8-f20f-47fe-93c8-e1ae4d2e5ef5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:19.287292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325e962-6905-4d3f-a794-cc926a43c448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:19.287888 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325e962-6905-4d3f-a794-cc926a43c448 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:19.288167 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325e962-6905-4d3f-a794-cc926a43c448 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:19.288444 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325e962-6905-4d3f-a794-cc926a43c448 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:19.289198 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325e962-6905-4d3f-a794-cc926a43c448 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:19.289500 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325e962-6905-4d3f-a794-cc926a43c448 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:19.290923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325e962-6905-4d3f-a794-cc926a43c448 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:19.294387 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d504e1-3393-4ed7-8b01-d99aa39f9cea None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:19.326450 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 196/400] 10.222.0.23 () {56 vars in 1137 bytes} [Thu Oct 20 13:23:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:19.327258 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325e962-6905-4d3f-a794-cc926a43c448 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:19.327390 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462084a5-5af4-4ba0-92a2-1605ec0d3466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:19.327977 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462084a5-5af4-4ba0-92a2-1605ec0d3466 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:19.328262 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462084a5-5af4-4ba0-92a2-1605ec0d3466 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:19.328533 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462084a5-5af4-4ba0-92a2-1605ec0d3466 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:19.336803 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462084a5-5af4-4ba0-92a2-1605ec0d3466 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:19.338705 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462084a5-5af4-4ba0-92a2-1605ec0d3466 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:19.351700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-462084a5-5af4-4ba0-92a2-1605ec0d3466 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:19.351700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:19.351700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:19.351700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:19.351700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:19.351700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:19.351700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:19.351700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:19.351700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:19.351700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:19.351700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:19.351700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:19.351700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:23:19.351700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:23:19.351700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:23:19.352766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:23:19.352766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:19.352766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:19.352766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:19.352766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:19.352766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:19.352766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:19.352766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:19.352766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:19.352766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:19.352766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:19.352766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:19.352766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:19.352766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:19.352766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:19.352766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:19.352766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:19.353662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:23:19.353662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:23:19.353662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:23:19.353662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:23:19.353662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:23:19.353662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:23:19.353662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:19.353662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:19.353662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 205/401] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:19.353662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462084a5-5af4-4ba0-92a2-1605ec0d3466 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:19.365185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:19.435891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325e962-6905-4d3f-a794-cc926a43c448 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:19.456134 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462084a5-5af4-4ba0-92a2-1605ec0d3466 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:19.459495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:19.472185 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5062096-a769-447b-9ff3-e396ef580e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:19.473001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5062096-a769-447b-9ff3-e396ef580e2c None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:19.473334 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5062096-a769-447b-9ff3-e396ef580e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:19.473685 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5062096-a769-447b-9ff3-e396ef580e2c None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:19.474336 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5062096-a769-447b-9ff3-e396ef580e2c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:19.475779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5062096-a769-447b-9ff3-e396ef580e2c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:19.489907 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da9352-0270-4b52-b51d-ec486eeaed3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:19.490698 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da9352-0270-4b52-b51d-ec486eeaed3c None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:19.491075 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da9352-0270-4b52-b51d-ec486eeaed3c None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:19.491454 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da9352-0270-4b52-b51d-ec486eeaed3c None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:19.492129 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da9352-0270-4b52-b51d-ec486eeaed3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:19.492485 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da9352-0270-4b52-b51d-ec486eeaed3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:19.493982 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da9352-0270-4b52-b51d-ec486eeaed3c None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:19.498460 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 197/402] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 20 13:23:19 2022] POST /identity/v3/users => generated 247 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:23:19.499261 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5062096-a769-447b-9ff3-e396ef580e2c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:19.522042 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 206/403] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:19.522761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da9352-0270-4b52-b51d-ec486eeaed3c None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:19.596848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:19.624334 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5062096-a769-447b-9ff3-e396ef580e2c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:19.639191 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da9352-0270-4b52-b51d-ec486eeaed3c None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:19.663086 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec502c3-0205-4bd1-9422-3dd2cb1f714e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:19.663753 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec502c3-0205-4bd1-9422-3dd2cb1f714e None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:19.664030 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec502c3-0205-4bd1-9422-3dd2cb1f714e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:19.664355 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec502c3-0205-4bd1-9422-3dd2cb1f714e None admin] PATH_INFO: `/v3/projects/4a81b6ecfb6e46a8b08c8bc2b4000fc7/users/ee5c2a3855cb4a668177aa19f337778d/roles/0f01a310371a4f6bbae7bb2e319a3b39` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:19.680129 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830c077-c031-4323-9e9b-f293f444147f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:19.680768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830c077-c031-4323-9e9b-f293f444147f None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:19.681087 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830c077-c031-4323-9e9b-f293f444147f None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:19.681269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec502c3-0205-4bd1-9422-3dd2cb1f714e None admin] RBAC: Authorizing `identity:create_grant(project_id=4a81b6ecfb6e46a8b08c8bc2b4000fc7, user_id=ee5c2a3855cb4a668177aa19f337778d, role_id=0f01a310371a4f6bbae7bb2e319a3b39)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:19.681366 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c830c077-c031-4323-9e9b-f293f444147f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:19.682794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec502c3-0205-4bd1-9422-3dd2cb1f714e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:19.693195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c830c077-c031-4323-9e9b-f293f444147f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:19.694761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c830c077-c031-4323-9e9b-f293f444147f None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:19.697313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 198/404] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:23:19 2022] PUT /identity/v3/projects/4a81b6ecfb6e46a8b08c8bc2b4000fc7/users/ee5c2a3855cb4a668177aa19f337778d/roles/0f01a310371a4f6bbae7bb2e319a3b39 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:23:19.698260 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec502c3-0205-4bd1-9422-3dd2cb1f714e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:19.711758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-c830c077-c031-4323-9e9b-f293f444147f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:19.711758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:19.711758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:19.711758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:19.711758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:19.711758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:19.711758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:19.711758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:19.711758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:19.711758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:19.711758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:19.711758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:19.711758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:19.711758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:19.711758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:19.711758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:19.712699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:19.712699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:19.712699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:19.712699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:19.712699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:19.712699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:19.712699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:19.712699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:19.712699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:19.712699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:19.712699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:19.712699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:19.712699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:19.712699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:19.712699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:19.712699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:19.712699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:19.714147 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:19.714147 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:19.714147 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:19.714147 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 207/405] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:19.714147 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830c077-c031-4323-9e9b-f293f444147f None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:19.748105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:19.837833 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec502c3-0205-4bd1-9422-3dd2cb1f714e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:19.876406 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c830c077-c031-4323-9e9b-f293f444147f None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:19.879417 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c10f8-0d74-4472-ad23-721bc9f889c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:19.880659 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c10f8-0d74-4472-ad23-721bc9f889c2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:19.882177 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c10f8-0d74-4472-ad23-721bc9f889c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:19.882177 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c10f8-0d74-4472-ad23-721bc9f889c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:19.882177 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c10f8-0d74-4472-ad23-721bc9f889c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:19.882412 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c10f8-0d74-4472-ad23-721bc9f889c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:19.884610 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c10f8-0d74-4472-ad23-721bc9f889c2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:19.899712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:19.905514 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 199/406] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 20 13:23:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:19.906489 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c10f8-0d74-4472-ad23-721bc9f889c2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:19.916010 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60050ee9-31bb-4e54-876a-d97175634cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:19.916641 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60050ee9-31bb-4e54-876a-d97175634cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:19.916950 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60050ee9-31bb-4e54-876a-d97175634cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:19.917227 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60050ee9-31bb-4e54-876a-d97175634cc1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:19.925385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60050ee9-31bb-4e54-876a-d97175634cc1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:19.927421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60050ee9-31bb-4e54-876a-d97175634cc1 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:19.940761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-60050ee9-31bb-4e54-876a-d97175634cc1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 20 13:23:19.940761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:19.940761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:19.940761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:19.940761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:19.940761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:19.940761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:19.940761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:19.940761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:19.940761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:19.940761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:19.940761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:19.940761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:23:19.940761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:23:19.940761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:23:19.940761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:23:19.941847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:19.941847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:19.941847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:23:19.941847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:19.941847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:23:19.941847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:19.941847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:19.941847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:19.941847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:19.941847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:19.941847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:19.941847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:19.941847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:19.941847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:19.941847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:19.941847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:19.941847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:19.941847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:19.943162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:19.943162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:19.943162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:23:19.943162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:23:19.943162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:23:19.943162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:23:19.943162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:23:19.943162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:23:19.943162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 20 13:23:19.943162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:19.943162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 208/407] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 20 13:23:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:19.943162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60050ee9-31bb-4e54-876a-d97175634cc1 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:20.022027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:20.034970 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c10f8-0d74-4472-ad23-721bc9f889c2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:20.061728 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60050ee9-31bb-4e54-876a-d97175634cc1 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:20.082468 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bfc0d5-6467-46d3-b990-23ddfaddce67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:20.084907 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bfc0d5-6467-46d3-b990-23ddfaddce67 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:20.085758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bfc0d5-6467-46d3-b990-23ddfaddce67 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:20.086179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bfc0d5-6467-46d3-b990-23ddfaddce67 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:20.087022 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bfc0d5-6467-46d3-b990-23ddfaddce67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:20.087487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bfc0d5-6467-46d3-b990-23ddfaddce67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:20.089035 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bfc0d5-6467-46d3-b990-23ddfaddce67 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:20.102248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 200/408] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:20.103358 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bfc0d5-6467-46d3-b990-23ddfaddce67 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:20.108784 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9867634-a931-4233-9d96-a6e0531a032e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:20.109606 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9867634-a931-4233-9d96-a6e0531a032e None admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:20.109980 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9867634-a931-4233-9d96-a6e0531a032e None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:20.110352 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9867634-a931-4233-9d96-a6e0531a032e None admin] PATH_INFO: `/v3/projects/4a81b6ecfb6e46a8b08c8bc2b4000fc7/users/87b5093e963c4753928a8c6091afc52b/roles/0f01a310371a4f6bbae7bb2e319a3b39` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:20.126277 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9867634-a931-4233-9d96-a6e0531a032e None admin] RBAC: Authorizing `identity:create_grant(project_id=4a81b6ecfb6e46a8b08c8bc2b4000fc7, user_id=87b5093e963c4753928a8c6091afc52b, role_id=0f01a310371a4f6bbae7bb2e319a3b39)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:20.127897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9867634-a931-4233-9d96-a6e0531a032e None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:20.149648 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 209/409] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:23:19 2022] PUT /identity/v3/projects/4a81b6ecfb6e46a8b08c8bc2b4000fc7/users/87b5093e963c4753928a8c6091afc52b/roles/0f01a310371a4f6bbae7bb2e319a3b39 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:23:20.149648 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9867634-a931-4233-9d96-a6e0531a032e None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:20.215687 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:20.268244 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bfc0d5-6467-46d3-b990-23ddfaddce67 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:20.309639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a62109-333c-432a-8e80-ad4f5bcffcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:20.310674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a62109-333c-432a-8e80-ad4f5bcffcae None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:20.311171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a62109-333c-432a-8e80-ad4f5bcffcae None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:20.311655 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a62109-333c-432a-8e80-ad4f5bcffcae None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:20.312608 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a62109-333c-432a-8e80-ad4f5bcffcae None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:20.313139 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a62109-333c-432a-8e80-ad4f5bcffcae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:20.314969 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a62109-333c-432a-8e80-ad4f5bcffcae None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:20.343980 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 201/410] 10.222.0.23 () {56 vars in 1139 bytes} [Thu Oct 20 13:23:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:20.350455 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a62109-333c-432a-8e80-ad4f5bcffcae None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:20.362766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9867634-a931-4233-9d96-a6e0531a032e None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:20.369238 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:20.403370 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca35fa-ca4a-4d95-97bf-b9de9df40e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:20.403999 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca35fa-ca4a-4d95-97bf-b9de9df40e97 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:20.404276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca35fa-ca4a-4d95-97bf-b9de9df40e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:20.404545 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca35fa-ca4a-4d95-97bf-b9de9df40e97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:20.412595 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca35fa-ca4a-4d95-97bf-b9de9df40e97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:20.414268 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca35fa-ca4a-4d95-97bf-b9de9df40e97 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:20.426015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-5bca35fa-ca4a-4d95-97bf-b9de9df40e97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:20.426015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:20.426015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:20.426015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:20.426015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:20.426015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:20.426015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:20.426015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:20.426015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:20.426015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:20.426015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:20.426015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:20.426015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:23:20.426015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:23:20.426015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:23:20.426921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:23:20.426921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:20.426921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:20.426921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:20.426921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:20.426921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:20.426921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:20.426921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:20.426921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:20.426921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:20.426921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:20.426921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:20.426921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:20.426921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:20.426921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:20.426921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:20.426921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:20.427741 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:23:20.427741 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:23:20.427741 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:23:20.427741 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:23:20.427741 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:23:20.427741 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:23:20.427741 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:20.427741 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:20.428134 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 210/411] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:20.434909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca35fa-ca4a-4d95-97bf-b9de9df40e97 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:20.468474 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a62109-333c-432a-8e80-ad4f5bcffcae None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:20.507383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43aa15c-9cd7-4e0a-82fd-f66937f9cddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:20.508028 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43aa15c-9cd7-4e0a-82fd-f66937f9cddd None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:20.508306 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43aa15c-9cd7-4e0a-82fd-f66937f9cddd None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:20.508577 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43aa15c-9cd7-4e0a-82fd-f66937f9cddd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:20.521313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43aa15c-9cd7-4e0a-82fd-f66937f9cddd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:20.522807 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43aa15c-9cd7-4e0a-82fd-f66937f9cddd None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:20.541572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-a43aa15c-9cd7-4e0a-82fd-f66937f9cddd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:20.541572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:20.541572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:20.541572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:20.541572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:20.541572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:20.541572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:20.541572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:20.541572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:20.541572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:20.541572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:20.541572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:20.541572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:20.541572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:20.541572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:20.541572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:20.542694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:20.542694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:20.542694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:20.542694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:20.542694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:20.542694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:20.542694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:20.542694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:20.542694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:20.542694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:20.542694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:20.542694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:20.542694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:20.542694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:20.542694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:20.542694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:20.542694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:20.543689 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:20.543689 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:20.543689 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:20.543689 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 202/412] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:20.553061 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43aa15c-9cd7-4e0a-82fd-f66937f9cddd None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:20.581314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:20.590897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca35fa-ca4a-4d95-97bf-b9de9df40e97 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:20.660153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8140d1f-a400-4450-aa44-af7cf3f40f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:20.660882 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8140d1f-a400-4450-aa44-af7cf3f40f00 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:20.661341 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8140d1f-a400-4450-aa44-af7cf3f40f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:20.661750 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8140d1f-a400-4450-aa44-af7cf3f40f00 None admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:20.662448 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8140d1f-a400-4450-aa44-af7cf3f40f00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:20.662757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8140d1f-a400-4450-aa44-af7cf3f40f00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:20.664201 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8140d1f-a400-4450-aa44-af7cf3f40f00 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:20.673710 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:20.696237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 211/413] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 20 13:23:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:20.701667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8140d1f-a400-4450-aa44-af7cf3f40f00 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:20.763212 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43aa15c-9cd7-4e0a-82fd-f66937f9cddd None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:20.821423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c017e6fc-8f5a-44aa-97f4-39925ca98181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:20.822044 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017e6fc-8f5a-44aa-97f4-39925ca98181 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:20.822330 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017e6fc-8f5a-44aa-97f4-39925ca98181 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:20.822609 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017e6fc-8f5a-44aa-97f4-39925ca98181 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:20.823257 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c017e6fc-8f5a-44aa-97f4-39925ca98181 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:20.823563 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c017e6fc-8f5a-44aa-97f4-39925ca98181 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:20.824993 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c017e6fc-8f5a-44aa-97f4-39925ca98181 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:20.839135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 203/414] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:20.847732 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c017e6fc-8f5a-44aa-97f4-39925ca98181 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:20.879101 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8140d1f-a400-4450-aa44-af7cf3f40f00 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:20.920144 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579d954-8247-4332-aa3e-5bdc4b335b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:20.920814 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579d954-8247-4332-aa3e-5bdc4b335b9f None admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:20.921479 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579d954-8247-4332-aa3e-5bdc4b335b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:20.921771 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579d954-8247-4332-aa3e-5bdc4b335b9f None admin] PATH_INFO: `/v3/projects/4a81b6ecfb6e46a8b08c8bc2b4000fc7/users/349ba417c3fc49ecaa11a65066dc54f8/roles/0f01a310371a4f6bbae7bb2e319a3b39` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:20.950007 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579d954-8247-4332-aa3e-5bdc4b335b9f None admin] RBAC: Authorizing `identity:create_grant(project_id=4a81b6ecfb6e46a8b08c8bc2b4000fc7, user_id=349ba417c3fc49ecaa11a65066dc54f8, role_id=0f01a310371a4f6bbae7bb2e319a3b39)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:20.951478 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579d954-8247-4332-aa3e-5bdc4b335b9f None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:20.966355 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:20.972419 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 212/415] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:23:20 2022] PUT /identity/v3/projects/4a81b6ecfb6e46a8b08c8bc2b4000fc7/users/349ba417c3fc49ecaa11a65066dc54f8/roles/0f01a310371a4f6bbae7bb2e319a3b39 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:23:21.013042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c017e6fc-8f5a-44aa-97f4-39925ca98181 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:21.047211 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:21.075554 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ce02b-d364-4256-98fc-57bc35c16e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:21.076268 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ce02b-d364-4256-98fc-57bc35c16e1b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:21.076545 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ce02b-d364-4256-98fc-57bc35c16e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:21.076818 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ce02b-d364-4256-98fc-57bc35c16e1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:21.087509 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ce02b-d364-4256-98fc-57bc35c16e1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:21.089120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ce02b-d364-4256-98fc-57bc35c16e1b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:21.103804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-b45ce02b-d364-4256-98fc-57bc35c16e1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:21.103804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:21.103804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:21.103804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:21.103804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:21.103804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:21.103804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:21.103804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:21.103804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:21.103804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:21.103804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:21.103804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:21.103804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:23:21.103804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:23:21.103804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:23:21.104880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:23:21.104880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:21.104880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:21.104880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:21.104880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:21.104880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:21.104880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:21.104880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:21.104880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:21.104880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:21.104880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:21.104880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:21.104880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:21.104880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:21.104880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:21.104880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:21.104880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:21.105826 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:23:21.105826 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:23:21.105826 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:23:21.105826 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:23:21.105826 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:23:21.105826 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:23:21.105826 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:21.105826 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:21.105826 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 204/416] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:21.114263 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579d954-8247-4332-aa3e-5bdc4b335b9f None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:21.248293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579d954-8247-4332-aa3e-5bdc4b335b9f None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:21.290552 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f413e-7ab4-489c-9c98-4fc3f07ed9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:21.291229 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f413e-7ab4-489c-9c98-4fc3f07ed9cd None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:21.291517 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f413e-7ab4-489c-9c98-4fc3f07ed9cd None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:21.291790 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f413e-7ab4-489c-9c98-4fc3f07ed9cd None admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:21.292491 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f413e-7ab4-489c-9c98-4fc3f07ed9cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:21.292799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f413e-7ab4-489c-9c98-4fc3f07ed9cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:21.294328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f413e-7ab4-489c-9c98-4fc3f07ed9cd None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:21.313373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 213/417] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 20 13:23:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:21.319722 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ce02b-d364-4256-98fc-57bc35c16e1b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:21.472628 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ce02b-d364-4256-98fc-57bc35c16e1b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:21.525584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92615945-69d8-4714-8e72-400ddfce44d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:21.526254 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92615945-69d8-4714-8e72-400ddfce44d7 None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:21.526533 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92615945-69d8-4714-8e72-400ddfce44d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:21.526813 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92615945-69d8-4714-8e72-400ddfce44d7 None admin] PATH_INFO: `/v3/projects/4a81b6ecfb6e46a8b08c8bc2b4000fc7/users/8a101adfc7a94bf1a4777350e9210a3a/roles/0f01a310371a4f6bbae7bb2e319a3b39` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:21.549568 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92615945-69d8-4714-8e72-400ddfce44d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a81b6ecfb6e46a8b08c8bc2b4000fc7, user_id=8a101adfc7a94bf1a4777350e9210a3a, role_id=0f01a310371a4f6bbae7bb2e319a3b39)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:21.551027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92615945-69d8-4714-8e72-400ddfce44d7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:21.580309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 205/418] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:23:21 2022] PUT /identity/v3/projects/4a81b6ecfb6e46a8b08c8bc2b4000fc7/users/8a101adfc7a94bf1a4777350e9210a3a/roles/0f01a310371a4f6bbae7bb2e319a3b39 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:23:21.982196 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bfb73-f0a5-4bb0-9885-e7201032317c None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:21.982491 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bfb73-f0a5-4bb0-9885-e7201032317c None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:21.982776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bfb73-f0a5-4bb0-9885-e7201032317c None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:21.984148 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 214/419] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:21.991854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f5c57-ff6a-4927-be5e-a1e6eb711de6 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:21.992275 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f5c57-ff6a-4927-be5e-a1e6eb711de6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:21.992648 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f5c57-ff6a-4927-be5e-a1e6eb711de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:22.012297 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-196f5c57-ff6a-4927-be5e-a1e6eb711de6 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:22.013860 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-196f5c57-ff6a-4927-be5e-a1e6eb711de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:22.039207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:22.099131 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 206/420] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:22.366361 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc34e5-fec2-43a3-bf33-8ee0e13ae0a6 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:22.366904 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc34e5-fec2-43a3-bf33-8ee0e13ae0a6 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:22.366904 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc34e5-fec2-43a3-bf33-8ee0e13ae0a6 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:22.368518 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 215/421] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:22.373800 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708a9ae-580c-4ab3-ad10-a33e56a47b5b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:22.374090 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708a9ae-580c-4ab3-ad10-a33e56a47b5b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:22.374364 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708a9ae-580c-4ab3-ad10-a33e56a47b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:22.393320 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c708a9ae-580c-4ab3-ad10-a33e56a47b5b None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:22.394515 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c708a9ae-580c-4ab3-ad10-a33e56a47b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:22.472752 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 207/422] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:22.486583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc34e5-fec2-43a3-bf33-8ee0e13ae0a6 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:22.573886 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:22.624136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc34e5-fec2-43a3-bf33-8ee0e13ae0a6 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:22.673573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee89ade-5b8e-40bf-95a8-d8f5bb80521e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:22.674219 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee89ade-5b8e-40bf-95a8-d8f5bb80521e None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:22.674502 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee89ade-5b8e-40bf-95a8-d8f5bb80521e None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:22.674776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee89ade-5b8e-40bf-95a8-d8f5bb80521e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:22.688515 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee89ade-5b8e-40bf-95a8-d8f5bb80521e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:22.694911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-0ee89ade-5b8e-40bf-95a8-d8f5bb80521e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:23:22.694911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:22.694911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:22.694911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:22.694911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:22.694911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:22.694911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:22.694911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:22.694911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:22.694911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:22.694911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:22.694911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:22.694911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:22.694911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:22.694911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:22.694911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:22.695841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:22.695841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:22.695841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:22.695841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:22.695841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:22.695841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:22.695841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:22.695841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:22.695841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:22.695841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:22.695841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:22.695841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:22.695841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:22.695841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:22.695841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:22.695841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:22.696622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:22.696622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:22.696622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:22.696622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:22.696622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:22.696622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:22.696622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:22.696622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:22.696622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:22.696622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:22.696622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:23:22.696622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:22.696622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 216/423] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 20 13:23:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:22.701812 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c708a9ae-580c-4ab3-ad10-a33e56a47b5b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:22.837614 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:22.838046 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee89ade-5b8e-40bf-95a8-d8f5bb80521e None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:22.848364 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c708a9ae-580c-4ab3-ad10-a33e56a47b5b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:22.905024 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8b3d0-8aa1-429f-8fae-8443ece0cecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:22.905562 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8b3d0-8aa1-429f-8fae-8443ece0cecf None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:22.905847 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8b3d0-8aa1-429f-8fae-8443ece0cecf None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:22.906126 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8b3d0-8aa1-429f-8fae-8443ece0cecf None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:22.906796 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8b3d0-8aa1-429f-8fae-8443ece0cecf None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:22.907518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8b3d0-8aa1-429f-8fae-8443ece0cecf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:22.908918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8b3d0-8aa1-429f-8fae-8443ece0cecf None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:22.919131 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 208/424] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 20 13:23:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:22.920832 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b05dd0-2fd0-4339-92fd-3fa61de9f342 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:22.921149 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b05dd0-2fd0-4339-92fd-3fa61de9f342 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:22.921429 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b05dd0-2fd0-4339-92fd-3fa61de9f342 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:22.923266 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 209/425] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:22.923997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b05dd0-2fd0-4339-92fd-3fa61de9f342 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:22.974011 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee89ade-5b8e-40bf-95a8-d8f5bb80521e None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:22.991823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:23.010656 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9cd94-3572-4d40-9e2e-65cdfe0c0ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:23.011503 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9cd94-3572-4d40-9e2e-65cdfe0c0ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:23.011920 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9cd94-3572-4d40-9e2e-65cdfe0c0ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:23.012326 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9cd94-3572-4d40-9e2e-65cdfe0c0ef4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:23.023610 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9cd94-3572-4d40-9e2e-65cdfe0c0ef4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:23.028945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-27d9cd94-3572-4d40-9e2e-65cdfe0c0ef4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:23:23.028945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:23.028945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:23.028945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:23.028945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:23.028945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:23.028945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:23.028945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:23.028945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:23.028945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:23.028945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:23.028945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:23.028945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:23.028945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:23.028945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:23.028945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:23.029989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:23.029989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:23.029989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:23.029989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:23.029989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:23.029989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:23.029989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:23.029989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:23.029989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:23.029989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:23.029989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:23.029989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:23.029989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:23.029989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:23.029989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:23.029989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:23.030909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:23.030909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:23.030909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:23.030909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:23.030909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:23.030909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:23.030909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:23.030909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:23.030909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:23.030909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:23.030909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:23:23.030909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:23.032933 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 217/426] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 20 13:23:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:23.037575 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b05dd0-2fd0-4339-92fd-3fa61de9f342 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:23.038940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d274ab-3cfc-4146-a00f-933eecea1ce2 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:23.039365 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d274ab-3cfc-4146-a00f-933eecea1ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:23.039774 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d274ab-3cfc-4146-a00f-933eecea1ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:23.058124 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-e0d274ab-3cfc-4146-a00f-933eecea1ce2 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:23.059475 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-e0d274ab-3cfc-4146-a00f-933eecea1ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:23.075671 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c728d70-0901-4cc0-a9a0-0347c2a6df50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:23.076295 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c728d70-0901-4cc0-a9a0-0347c2a6df50 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:23.076617 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c728d70-0901-4cc0-a9a0-0347c2a6df50 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:23.076926 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c728d70-0901-4cc0-a9a0-0347c2a6df50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:23.090574 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c728d70-0901-4cc0-a9a0-0347c2a6df50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:23.092025 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c728d70-0901-4cc0-a9a0-0347c2a6df50 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:23.109142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-3c728d70-0901-4cc0-a9a0-0347c2a6df50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:23.109142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:23.109142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:23.109142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:23.109142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:23.109142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:23.109142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:23.109142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:23.109142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:23.109142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:23.109142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:23.109142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:23.109142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:23.109142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:23.109142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:23.109142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:23.110582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:23.110582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:23.110582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:23.110582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:23.110582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:23.110582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:23.110582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:23.110582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:23.110582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:23.110582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:23.110582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:23.110582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:23.110582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:23.110582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:23.110582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:23.110582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:23.110582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:23.111556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:23.111556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:23.111556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:23.111556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 210/427] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:23.111556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c728d70-0901-4cc0-a9a0-0347c2a6df50 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:23.115421 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:23.135266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 218/428] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:23.136107 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d274ab-3cfc-4146-a00f-933eecea1ce2 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:23.227177 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c728d70-0901-4cc0-a9a0-0347c2a6df50 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:23.265789 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59156955-341e-4801-8b46-c6feaff4a5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:23.266639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59156955-341e-4801-8b46-c6feaff4a5d5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:23.267067 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59156955-341e-4801-8b46-c6feaff4a5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:23.267511 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59156955-341e-4801-8b46-c6feaff4a5d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:23.268322 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59156955-341e-4801-8b46-c6feaff4a5d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:23.268788 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59156955-341e-4801-8b46-c6feaff4a5d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:23.270498 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59156955-341e-4801-8b46-c6feaff4a5d5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:23.280468 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 211/429] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 20 13:23:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:23.294459 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59156955-341e-4801-8b46-c6feaff4a5d5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:23.337936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d274ab-3cfc-4146-a00f-933eecea1ce2 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:23.352090 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:23.371444 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:23.394251 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120882df-3386-4c7d-afb7-cce7c7ceeb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:23.394888 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120882df-3386-4c7d-afb7-cce7c7ceeb00 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:23.395178 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120882df-3386-4c7d-afb7-cce7c7ceeb00 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:23.395436 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120882df-3386-4c7d-afb7-cce7c7ceeb00 None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:23.396184 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120882df-3386-4c7d-afb7-cce7c7ceeb00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:23.396489 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120882df-3386-4c7d-afb7-cce7c7ceeb00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:23.397995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120882df-3386-4c7d-afb7-cce7c7ceeb00 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:23.410527 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 219/430] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:23.413474 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59156955-341e-4801-8b46-c6feaff4a5d5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:23.416319 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120882df-3386-4c7d-afb7-cce7c7ceeb00 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:23.453725 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119f368-a277-4217-b256-86a27b10aefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:23.454491 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119f368-a277-4217-b256-86a27b10aefd None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:23.454780 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119f368-a277-4217-b256-86a27b10aefd None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:23.455057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119f368-a277-4217-b256-86a27b10aefd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:23.468897 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119f368-a277-4217-b256-86a27b10aefd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:23.470543 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119f368-a277-4217-b256-86a27b10aefd None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:23.487286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-f119f368-a277-4217-b256-86a27b10aefd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:23.487286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:23.487286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:23.487286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:23.487286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:23.487286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:23.487286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:23.487286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:23.487286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:23.487286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:23.487286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:23.487286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:23.487286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:23.487286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:23.487286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:23.487286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:23.488162 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:23.488162 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:23.488162 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:23.488162 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:23.488162 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:23.488162 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:23.488162 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:23.488162 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:23.488162 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:23.488162 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:23.488162 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:23.488162 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:23.488162 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:23.488162 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:23.488162 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:23.488162 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:23.488162 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:23.489627 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:23.489627 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:23.489627 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:23.492940 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 212/431] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:23:23.494979 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119f368-a277-4217-b256-86a27b10aefd None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:23.537172 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120882df-3386-4c7d-afb7-cce7c7ceeb00 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:23.587225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d721c9f-b3bb-4c34-8922-e631cbbdd828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:23.587884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d721c9f-b3bb-4c34-8922-e631cbbdd828 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:23.588258 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d721c9f-b3bb-4c34-8922-e631cbbdd828 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:23.588429 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d721c9f-b3bb-4c34-8922-e631cbbdd828 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:23.596541 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d721c9f-b3bb-4c34-8922-e631cbbdd828 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:23.598325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d721c9f-b3bb-4c34-8922-e631cbbdd828 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:23.610721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-3d721c9f-b3bb-4c34-8922-e631cbbdd828 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 20 13:23:23.610721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:23.610721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:23.610721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:23.610721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:23.610721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:23.610721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:23.610721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:23.610721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:23.610721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:23.610721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:23.610721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:23.610721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:23:23.610721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:23:23.610721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:23:23.610721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:23:23.611700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:23.611700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:23.611700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:23:23.611700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:23.611700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:23:23.611700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:23.611700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:23.611700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:23.611700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:23.611700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:23.611700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:23.611700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:23.611700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:23.611700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:23.611700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:23.611700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:23.611700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:23.611700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:23.613195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:23.613195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:23.613195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:23:23.613195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:23:23.613195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:23:23.613195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:23:23.613195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:23:23.613195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:23:23.613195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 20 13:23:23.613195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:23.613195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 220/432] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 20 13:23:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:23.613195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d721c9f-b3bb-4c34-8922-e631cbbdd828 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:23.615891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119f368-a277-4217-b256-86a27b10aefd None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:23.635522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:23.647272 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:23.658003 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737e623-6a61-4390-8ca9-4074611f7780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:23.659008 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737e623-6a61-4390-8ca9-4074611f7780 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:23.659529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737e623-6a61-4390-8ca9-4074611f7780 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:23.660106 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737e623-6a61-4390-8ca9-4074611f7780 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:23.661048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2737e623-6a61-4390-8ca9-4074611f7780 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:23.661553 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2737e623-6a61-4390-8ca9-4074611f7780 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:23.663260 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2737e623-6a61-4390-8ca9-4074611f7780 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:23.679119 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 213/433] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:23.680302 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737e623-6a61-4390-8ca9-4074611f7780 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:23.732594 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d721c9f-b3bb-4c34-8922-e631cbbdd828 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:23.772158 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16fd6cc-a324-44ce-b74e-08008abdeb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:23.772734 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fd6cc-a324-44ce-b74e-08008abdeb92 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:23.773065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fd6cc-a324-44ce-b74e-08008abdeb92 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:23.773337 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fd6cc-a324-44ce-b74e-08008abdeb92 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:23.784853 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16fd6cc-a324-44ce-b74e-08008abdeb92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:23.791458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-d16fd6cc-a324-44ce-b74e-08008abdeb92 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:23:23.791458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:23.791458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:23.791458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:23.791458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:23.791458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:23.791458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:23.791458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:23.791458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:23.791458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:23.791458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:23.791458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:23.791458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:23.791458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:23.791458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:23.791458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:23.792351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:23.792351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:23.792351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:23.792351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:23.792351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:23.792351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:23.792351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:23.792351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:23.792351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:23.792351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:23.792351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:23.792351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:23.792351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:23.792351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:23.792351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:23.792351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:23.793162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:23.793162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:23.793162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:23.793162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:23.793162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:23.793162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:23.793162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:23.793162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:23.793162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:23.793162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:23.793162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:23:23.793162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:23.797616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 221/434] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 20 13:23:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:23.798401 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16fd6cc-a324-44ce-b74e-08008abdeb92 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:23.807339 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737e623-6a61-4390-8ca9-4074611f7780 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:23.855855 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60eff4-6c0c-4498-90fc-7064c96df163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:23.857009 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60eff4-6c0c-4498-90fc-7064c96df163 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:23.857295 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60eff4-6c0c-4498-90fc-7064c96df163 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:23.857570 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60eff4-6c0c-4498-90fc-7064c96df163 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:23.858681 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60eff4-6c0c-4498-90fc-7064c96df163 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:23.858996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60eff4-6c0c-4498-90fc-7064c96df163 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:23.860449 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60eff4-6c0c-4498-90fc-7064c96df163 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:23.881165 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 214/435] 10.222.0.23 () {56 vars in 1135 bytes} [Thu Oct 20 13:23:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:23.882135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60eff4-6c0c-4498-90fc-7064c96df163 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:23.911612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:23.923862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:23.929242 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16fd6cc-a324-44ce-b74e-08008abdeb92 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:23.964500 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3aa9f0-aa7f-43f7-8e98-3249aa683ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:23.965175 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3aa9f0-aa7f-43f7-8e98-3249aa683ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:23.965456 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3aa9f0-aa7f-43f7-8e98-3249aa683ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:23.965726 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3aa9f0-aa7f-43f7-8e98-3249aa683ea9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:23.973897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3aa9f0-aa7f-43f7-8e98-3249aa683ea9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:23.975530 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3aa9f0-aa7f-43f7-8e98-3249aa683ea9 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:23.987228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-3b3aa9f0-aa7f-43f7-8e98-3249aa683ea9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 20 13:23:23.987228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:23.987228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:23.987228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:23.987228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:23.987228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:23.987228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:23.987228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:23.987228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:23.987228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:23.987228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:23.987228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:23.987228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:23:23.987228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:23:23.987228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:23:23.987228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:23:23.988117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:23.988117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:23.988117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:23:23.988117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:23.988117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:23:23.988117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:23.988117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:23.988117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:23.988117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:23.988117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:23.988117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:23.988117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:23.988117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:23.988117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:23.988117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:23.988117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:23.988117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:23.988117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:23.989907 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:23.989907 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:23.989907 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:23:23.989907 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:23:23.989907 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:23:23.989907 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:23:23.989907 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:23:23.989907 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:23:23.989907 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 20 13:23:23.989907 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:23.989907 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 222/436] 10.222.0.23 () {56 vars in 1093 bytes} [Thu Oct 20 13:23:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:23.990489 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3aa9f0-aa7f-43f7-8e98-3249aa683ea9 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:24.005961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60eff4-6c0c-4498-90fc-7064c96df163 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:24.043846 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5f867-1038-4f3b-aea3-cc122380afde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:24.044770 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5f867-1038-4f3b-aea3-cc122380afde None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:24.045295 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5f867-1038-4f3b-aea3-cc122380afde None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:24.045815 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5f867-1038-4f3b-aea3-cc122380afde None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:24.046715 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5f867-1038-4f3b-aea3-cc122380afde None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:24.047197 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5f867-1038-4f3b-aea3-cc122380afde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:24.048920 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5f867-1038-4f3b-aea3-cc122380afde None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:24.059195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 215/437] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 20 13:23:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:24.060844 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d6abb-46b1-4218-a669-e86b4ab6e68c None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:24.061173 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d6abb-46b1-4218-a669-e86b4ab6e68c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:24.061450 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d6abb-46b1-4218-a669-e86b4ab6e68c None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:24.062816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 216/438] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:24.063425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d6abb-46b1-4218-a669-e86b4ab6e68c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:24.106516 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3aa9f0-aa7f-43f7-8e98-3249aa683ea9 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:24.140525 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6fb6f-5d26-42d3-91ed-a9f24522b8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:24.141185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6fb6f-5d26-42d3-91ed-a9f24522b8ca None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:24.141459 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6fb6f-5d26-42d3-91ed-a9f24522b8ca None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:24.141727 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6fb6f-5d26-42d3-91ed-a9f24522b8ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:24.153012 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6fb6f-5d26-42d3-91ed-a9f24522b8ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:24.154484 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6fb6f-5d26-42d3-91ed-a9f24522b8ca None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:24.163185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:24.170188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-c6b6fb6f-5d26-42d3-91ed-a9f24522b8ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:24.170188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:24.170188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:24.170188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:24.170188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:24.170188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:24.170188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:24.170188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:24.170188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:24.170188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:24.170188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:24.170188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:24.170188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:24.170188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:24.170188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:24.170188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:24.171074 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:24.171074 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:24.171074 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:24.171074 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:24.171074 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:24.171074 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:24.171074 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:24.171074 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:24.171074 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:24.171074 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:24.171074 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:24.171074 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:24.171074 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:24.171074 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:24.171074 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:24.171074 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:24.171074 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:24.171920 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:24.171920 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:24.171920 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:24.173518 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 223/439] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:24.174432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6fb6f-5d26-42d3-91ed-a9f24522b8ca None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:24.187183 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:24.193504 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d6abb-46b1-4218-a669-e86b4ab6e68c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:24.227439 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269c31b-9b7b-47d5-a34e-770db8ee5fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:24.228043 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269c31b-9b7b-47d5-a34e-770db8ee5fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:24.228837 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269c31b-9b7b-47d5-a34e-770db8ee5fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:24.228837 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269c31b-9b7b-47d5-a34e-770db8ee5fd0 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:24.229355 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269c31b-9b7b-47d5-a34e-770db8ee5fd0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:24.229659 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269c31b-9b7b-47d5-a34e-770db8ee5fd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:24.231145 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269c31b-9b7b-47d5-a34e-770db8ee5fd0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:24.248549 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 217/440] 10.222.0.23 () {56 vars in 1133 bytes} [Thu Oct 20 13:23:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:24.249798 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f726b5-99b5-45b3-babc-9eef81d16c44 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:24.250085 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f726b5-99b5-45b3-babc-9eef81d16c44 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:24.250365 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f726b5-99b5-45b3-babc-9eef81d16c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:24.269042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-44f726b5-99b5-45b3-babc-9eef81d16c44 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:24.271076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-44f726b5-99b5-45b3-babc-9eef81d16c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:24.285168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6fb6f-5d26-42d3-91ed-a9f24522b8ca None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:24.319191 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee939d-a8a1-4e26-a756-eb4e86dfdfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:24.319813 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee939d-a8a1-4e26-a756-eb4e86dfdfc5 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:24.320104 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee939d-a8a1-4e26-a756-eb4e86dfdfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:24.320378 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee939d-a8a1-4e26-a756-eb4e86dfdfc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:24.330675 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee939d-a8a1-4e26-a756-eb4e86dfdfc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:24.332125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee939d-a8a1-4e26-a756-eb4e86dfdfc5 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:24.336853 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 218/441] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:24.337536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f726b5-99b5-45b3-babc-9eef81d16c44 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:24.346447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-f0ee939d-a8a1-4e26-a756-eb4e86dfdfc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:24.346447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:24.346447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:24.346447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:24.346447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:24.346447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:24.346447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:24.346447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:24.346447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:24.346447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:24.346447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:24.346447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:24.346447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:24.346447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:24.346447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:24.346447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:24.347293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:24.347293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:24.347293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:24.347293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:24.347293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:24.347293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:24.347293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:24.347293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:24.347293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:24.347293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:24.347293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:24.347293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:24.347293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:24.347293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:24.347293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:24.347293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:24.347293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:24.348193 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:24.348193 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:24.348193 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:24.348193 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 224/442] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:24.348691 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6041bb0-cf0c-4cf8-afe0-cdd270d701a7 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:24.348988 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6041bb0-cf0c-4cf8-afe0-cdd270d701a7 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:24.349266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6041bb0-cf0c-4cf8-afe0-cdd270d701a7 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:24.352245 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 225/443] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:24.352245 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6041bb0-cf0c-4cf8-afe0-cdd270d701a7 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:24.411230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:24.439373 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:24.456849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f726b5-99b5-45b3-babc-9eef81d16c44 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:24.462556 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6041bb0-cf0c-4cf8-afe0-cdd270d701a7 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:24.494868 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1efcf-44c2-441d-ab68-e6b227658574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:24.495475 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1efcf-44c2-441d-ab68-e6b227658574 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:24.495755 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1efcf-44c2-441d-ab68-e6b227658574 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:24.496026 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1efcf-44c2-441d-ab68-e6b227658574 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:24.496669 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1efcf-44c2-441d-ab68-e6b227658574 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:24.496998 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1efcf-44c2-441d-ab68-e6b227658574 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:24.498387 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1efcf-44c2-441d-ab68-e6b227658574 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:24.501407 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2a3c6-6969-4337-a169-b9883c75cd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:24.502269 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2a3c6-6969-4337-a169-b9883c75cd8d None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:24.502745 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2a3c6-6969-4337-a169-b9883c75cd8d None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:24.503213 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2a3c6-6969-4337-a169-b9883c75cd8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:24.511426 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 219/444] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:24.512385 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1efcf-44c2-441d-ab68-e6b227658574 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:24.578277 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2a3c6-6969-4337-a169-b9883c75cd8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:24.579656 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2a3c6-6969-4337-a169-b9883c75cd8d None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:24.621717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1efcf-44c2-441d-ab68-e6b227658574 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:24.633497 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:24.655554 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4d5e3-211a-4b1c-9df1-4148ceb3bf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:24.656173 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4d5e3-211a-4b1c-9df1-4148ceb3bf2c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:24.656456 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4d5e3-211a-4b1c-9df1-4148ceb3bf2c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:24.656733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4d5e3-211a-4b1c-9df1-4148ceb3bf2c None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:24.657404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4d5e3-211a-4b1c-9df1-4148ceb3bf2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:24.657711 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4d5e3-211a-4b1c-9df1-4148ceb3bf2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:24.659079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4d5e3-211a-4b1c-9df1-4148ceb3bf2c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:24.670731 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 220/445] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:24.675304 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c742e-9411-4511-b159-3e7b4b6e113a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:24.675304 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c742e-9411-4511-b159-3e7b4b6e113a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:24.675304 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c742e-9411-4511-b159-3e7b4b6e113a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:24.684718 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:24.685534 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 226/446] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:23:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 13:23:24.686272 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2a3c6-6969-4337-a169-b9883c75cd8d None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:24.694460 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1f0c742e-9411-4511-b159-3e7b4b6e113a None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:24.695654 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1f0c742e-9411-4511-b159-3e7b4b6e113a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:24.762965 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 221/447] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:24.764049 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c742e-9411-4511-b159-3e7b4b6e113a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:24.783690 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2a3c6-6969-4337-a169-b9883c75cd8d None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:24.819681 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a913d8e-e5fb-4242-a1d2-d827bcf922f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:24.820303 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a913d8e-e5fb-4242-a1d2-d827bcf922f4 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:24.820581 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a913d8e-e5fb-4242-a1d2-d827bcf922f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:24.820852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a913d8e-e5fb-4242-a1d2-d827bcf922f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:24.827679 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a913d8e-e5fb-4242-a1d2-d827bcf922f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:24.829225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a913d8e-e5fb-4242-a1d2-d827bcf922f4 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:24.840951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-0a913d8e-e5fb-4242-a1d2-d827bcf922f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:24.840951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:24.840951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:24.840951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:24.840951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:24.840951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:24.840951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:24.840951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:24.840951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:24.840951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:24.840951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:24.840951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:24.840951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:23:24.840951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:23:24.840951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:23:24.841770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:23:24.841770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:24.841770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:24.841770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:24.841770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:24.841770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:24.841770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:24.841770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:24.841770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:24.841770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:24.841770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:24.841770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:24.841770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:24.841770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:24.841770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:24.841770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:24.841770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:24.843170 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:23:24.843170 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:23:24.843170 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:23:24.843170 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:23:24.843170 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:23:24.843170 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:23:24.843170 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:24.843170 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:24.843170 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 227/448] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:24.843170 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a913d8e-e5fb-4242-a1d2-d827bcf922f4 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:24.858638 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:24.887117 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c742e-9411-4511-b159-3e7b4b6e113a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:24.923212 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d7b6f-9c93-4244-992e-70b2227a89f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:24.923829 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d7b6f-9c93-4244-992e-70b2227a89f9 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:24.924108 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d7b6f-9c93-4244-992e-70b2227a89f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:24.924380 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d7b6f-9c93-4244-992e-70b2227a89f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:24.927518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:24.936160 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d7b6f-9c93-4244-992e-70b2227a89f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:24.940994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-406d7b6f-9c93-4244-992e-70b2227a89f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:23:24.940994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:24.940994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:24.940994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:24.940994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:24.940994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:24.940994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:24.940994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:24.940994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:24.940994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:24.940994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:24.940994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:24.940994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:24.940994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:24.940994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:24.940994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:24.942692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:24.942692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:24.942692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:24.942692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:24.942692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:24.942692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:24.942692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:24.942692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:24.942692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:24.942692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:24.942692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:24.942692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:24.942692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:24.942692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:24.942692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:24.942692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:24.943600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:24.943600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:24.943600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:24.943600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:24.943600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:24.943600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:24.943600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:24.943600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:24.943600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:24.943600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:24.943600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:23:24.943600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:24.943600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 222/449] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 20 13:23:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:24.946237 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d7b6f-9c93-4244-992e-70b2227a89f9 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:24.951239 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a913d8e-e5fb-4242-a1d2-d827bcf922f4 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:24.987099 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022939b-735d-423b-9adb-8d7f80fe5f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:24.987729 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022939b-735d-423b-9adb-8d7f80fe5f6c None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:24.988013 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022939b-735d-423b-9adb-8d7f80fe5f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:24.988288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022939b-735d-423b-9adb-8d7f80fe5f6c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:24.995567 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6022939b-735d-423b-9adb-8d7f80fe5f6c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:24.997054 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6022939b-735d-423b-9adb-8d7f80fe5f6c None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:25.009135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-6022939b-735d-423b-9adb-8d7f80fe5f6c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 20 13:23:25.009135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:25.009135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:25.009135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:25.009135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:25.009135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:25.009135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:25.009135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:25.009135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:25.009135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:25.009135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:25.009135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:25.009135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:23:25.009135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:23:25.009135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:23:25.009135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:23:25.010034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:25.010034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:25.010034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:23:25.010034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:25.010034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:23:25.010034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:25.010034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:25.010034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:25.010034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:25.010034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:25.010034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:25.010034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:25.010034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:25.010034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:25.010034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:25.010034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:25.010034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:25.010034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:25.010909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:25.010909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:25.010909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:23:25.010909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:23:25.010909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:23:25.010909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:23:25.010909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:23:25.010909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:23:25.010909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 20 13:23:25.010909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:25.010909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 228/450] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:23:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:25.011525 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022939b-735d-423b-9adb-8d7f80fe5f6c None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:25.055122 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d7b6f-9c93-4244-992e-70b2227a89f9 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:25.084660 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:25.090632 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d540da6-ee57-4970-b642-618c91a44098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:25.091256 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d540da6-ee57-4970-b642-618c91a44098 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:25.091533 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d540da6-ee57-4970-b642-618c91a44098 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:25.091803 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d540da6-ee57-4970-b642-618c91a44098 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:25.102281 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d540da6-ee57-4970-b642-618c91a44098 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:25.103710 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d540da6-ee57-4970-b642-618c91a44098 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:25.114810 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022939b-735d-423b-9adb-8d7f80fe5f6c None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:25.118779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-8d540da6-ee57-4970-b642-618c91a44098 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:25.118779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:25.118779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:25.118779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:25.118779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:25.118779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:25.118779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:25.118779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:25.118779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:25.118779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:25.118779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:25.118779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:25.118779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:25.118779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:25.118779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:25.118779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:25.119674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:25.119674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:25.119674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:25.119674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:25.119674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:25.119674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:25.119674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:25.119674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:25.119674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:25.119674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:25.119674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:25.119674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:25.119674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:25.119674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:25.119674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:25.119674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:25.119674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:25.120497 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:25.120497 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:25.120497 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:25.120497 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 223/451] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:25.120723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d540da6-ee57-4970-b642-618c91a44098 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:25.151884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb2888-8bda-4833-bc7a-5e429e2d1262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:25.152474 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb2888-8bda-4833-bc7a-5e429e2d1262 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:25.152751 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb2888-8bda-4833-bc7a-5e429e2d1262 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:25.153045 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb2888-8bda-4833-bc7a-5e429e2d1262 None admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:25.153724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb2888-8bda-4833-bc7a-5e429e2d1262 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:25.154025 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb2888-8bda-4833-bc7a-5e429e2d1262 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:25.155446 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb2888-8bda-4833-bc7a-5e429e2d1262 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:25.166847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 229/452] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 20 13:23:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:25.167621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb2888-8bda-4833-bc7a-5e429e2d1262 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:25.174499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:25.225041 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d540da6-ee57-4970-b642-618c91a44098 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:25.259074 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e2e26-cc33-46f4-997b-4d5427d8939e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:25.259702 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e2e26-cc33-46f4-997b-4d5427d8939e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:25.259984 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e2e26-cc33-46f4-997b-4d5427d8939e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:25.260258 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e2e26-cc33-46f4-997b-4d5427d8939e None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:25.260946 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e2e26-cc33-46f4-997b-4d5427d8939e None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:25.261251 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e2e26-cc33-46f4-997b-4d5427d8939e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:25.262913 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e2e26-cc33-46f4-997b-4d5427d8939e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:25.264487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb2888-8bda-4833-bc7a-5e429e2d1262 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:25.270547 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 224/453] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 20 13:23:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:25.271327 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e2e26-cc33-46f4-997b-4d5427d8939e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:25.293255 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:25.298939 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e6f7df-c1fe-4c74-a8be-544053faea51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:25.299886 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6f7df-c1fe-4c74-a8be-544053faea51 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:25.300168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6f7df-c1fe-4c74-a8be-544053faea51 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:25.300441 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6f7df-c1fe-4c74-a8be-544053faea51 None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:25.301739 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e6f7df-c1fe-4c74-a8be-544053faea51 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:25.302065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e6f7df-c1fe-4c74-a8be-544053faea51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:25.303496 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e6f7df-c1fe-4c74-a8be-544053faea51 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:25.320615 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 230/454] 10.222.0.23 () {56 vars in 1137 bytes} [Thu Oct 20 13:23:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:25.321448 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e6f7df-c1fe-4c74-a8be-544053faea51 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:25.370850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e2e26-cc33-46f4-997b-4d5427d8939e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:25.381787 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:25.405351 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495ee40-b46b-46d3-8c47-79e03a368075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:25.405974 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495ee40-b46b-46d3-8c47-79e03a368075 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:25.406253 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495ee40-b46b-46d3-8c47-79e03a368075 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:25.406525 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495ee40-b46b-46d3-8c47-79e03a368075 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:25.417435 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495ee40-b46b-46d3-8c47-79e03a368075 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:25.422849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-7495ee40-b46b-46d3-8c47-79e03a368075 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:23:25.422849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:25.422849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:25.422849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:25.422849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:25.422849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:25.422849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:25.422849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:25.422849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:25.422849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:25.422849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:25.422849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:25.422849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:25.422849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:25.422849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:25.422849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:25.423905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:25.423905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:25.423905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:25.423905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:25.423905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:25.423905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:25.423905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:25.423905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:25.423905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:25.423905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:25.423905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:25.423905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:25.423905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:25.423905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:25.423905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:25.423905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:25.424906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:25.424906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:25.424906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:25.424906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:25.424906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:25.424906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:25.424906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:25.424906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:25.424906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:25.424906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:25.424906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:23:25.424906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:25.425516 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e6f7df-c1fe-4c74-a8be-544053faea51 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:25.425912 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 225/455] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 20 13:23:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:25.425912 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495ee40-b46b-46d3-8c47-79e03a368075 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:25.458002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658aa66c-b5b4-43d9-8e50-e306c7df3863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:25.458596 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658aa66c-b5b4-43d9-8e50-e306c7df3863 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:25.458875 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658aa66c-b5b4-43d9-8e50-e306c7df3863 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:25.459149 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658aa66c-b5b4-43d9-8e50-e306c7df3863 None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:25.459786 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658aa66c-b5b4-43d9-8e50-e306c7df3863 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:25.460091 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658aa66c-b5b4-43d9-8e50-e306c7df3863 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:25.461504 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658aa66c-b5b4-43d9-8e50-e306c7df3863 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:25.472667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 231/456] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:25.473806 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658aa66c-b5b4-43d9-8e50-e306c7df3863 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:25.526423 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:25.531959 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495ee40-b46b-46d3-8c47-79e03a368075 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:25.565352 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adde9fd-bd5b-439a-a41e-3f1681f8d016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:25.565999 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adde9fd-bd5b-439a-a41e-3f1681f8d016 None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:25.566277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adde9fd-bd5b-439a-a41e-3f1681f8d016 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:25.566548 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adde9fd-bd5b-439a-a41e-3f1681f8d016 None admin] PATH_INFO: `/v3/projects/4a81b6ecfb6e46a8b08c8bc2b4000fc7/users/9834211488354651803000c7e8fab80b/roles/0f01a310371a4f6bbae7bb2e319a3b39` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:25.576637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658aa66c-b5b4-43d9-8e50-e306c7df3863 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:25.580316 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adde9fd-bd5b-439a-a41e-3f1681f8d016 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a81b6ecfb6e46a8b08c8bc2b4000fc7, user_id=9834211488354651803000c7e8fab80b, role_id=0f01a310371a4f6bbae7bb2e319a3b39)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:25.581797 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adde9fd-bd5b-439a-a41e-3f1681f8d016 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:25.594234 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 226/457] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:23:25 2022] PUT /identity/v3/projects/4a81b6ecfb6e46a8b08c8bc2b4000fc7/users/9834211488354651803000c7e8fab80b/roles/0f01a310371a4f6bbae7bb2e319a3b39 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:23:25.595032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adde9fd-bd5b-439a-a41e-3f1681f8d016 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:25.611135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e7321-7b9e-42ad-96af-c8a9c1ed6720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:25.611786 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e7321-7b9e-42ad-96af-c8a9c1ed6720 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:25.612065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e7321-7b9e-42ad-96af-c8a9c1ed6720 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:25.612341 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e7321-7b9e-42ad-96af-c8a9c1ed6720 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:25.625441 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e7321-7b9e-42ad-96af-c8a9c1ed6720 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:25.627130 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e7321-7b9e-42ad-96af-c8a9c1ed6720 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:25.631165 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:25.645376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-8a3e7321-7b9e-42ad-96af-c8a9c1ed6720 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:25.645376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:25.645376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:25.645376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:25.645376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:25.645376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:25.645376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:25.645376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:25.645376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:25.645376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:25.645376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:25.645376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:25.645376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:25.645376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:25.645376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:25.645376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:25.646377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:25.646377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:25.646377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:25.646377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:25.646377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:25.646377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:25.646377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:25.646377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:25.646377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:25.646377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:25.646377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:25.646377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:25.646377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:25.646377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:25.646377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:25.646377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:25.646377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:25.647197 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:25.647197 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:25.647197 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:25.647197 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 232/458] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:25.647995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e7321-7b9e-42ad-96af-c8a9c1ed6720 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:25.729855 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adde9fd-bd5b-439a-a41e-3f1681f8d016 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:25.763732 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e7321-7b9e-42ad-96af-c8a9c1ed6720 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:25.765409 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:25.771626 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c019abca-3e7b-4132-bd4b-a87dbe4a03c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:25.772238 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019abca-3e7b-4132-bd4b-a87dbe4a03c2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:25.773029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019abca-3e7b-4132-bd4b-a87dbe4a03c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:25.773408 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019abca-3e7b-4132-bd4b-a87dbe4a03c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:25.801775 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8b24f-3f76-44ea-aac8-422a83cfb018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:25.802694 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8b24f-3f76-44ea-aac8-422a83cfb018 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:25.803195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8b24f-3f76-44ea-aac8-422a83cfb018 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:25.803465 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8b24f-3f76-44ea-aac8-422a83cfb018 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:25.804132 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a8b24f-3f76-44ea-aac8-422a83cfb018 None admin] RBAC: Adding query filter params (name=service) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:25.804431 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a8b24f-3f76-44ea-aac8-422a83cfb018 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:25.805890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a8b24f-3f76-44ea-aac8-422a83cfb018 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:25.818865 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 233/459] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 20 13:23:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:25.818865 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8b24f-3f76-44ea-aac8-422a83cfb018 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:25.856773 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c019abca-3e7b-4132-bd4b-a87dbe4a03c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:25.858596 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c019abca-3e7b-4132-bd4b-a87dbe4a03c2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:25.885005 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:25.920162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8b24f-3f76-44ea-aac8-422a83cfb018 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:25.954950 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930a2ad-aee4-4100-b51d-a41604694ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:25.955576 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930a2ad-aee4-4100-b51d-a41604694ba0 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:25.955854 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930a2ad-aee4-4100-b51d-a41604694ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:25.956129 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930a2ad-aee4-4100-b51d-a41604694ba0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:25.962920 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930a2ad-aee4-4100-b51d-a41604694ba0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:25.964374 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930a2ad-aee4-4100-b51d-a41604694ba0 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:25.972590 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 227/460] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:23:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 13:23:25.975915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-d930a2ad-aee4-4100-b51d-a41604694ba0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:25.975915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:25.975915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:25.975915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:25.975915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:25.975915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:25.975915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:25.975915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:25.975915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:25.975915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:25.975915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:25.975915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:25.975915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:23:25.975915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:23:25.975915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:23:25.976798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:23:25.976798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:25.976798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:25.976798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:25.976798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:25.976798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:25.976798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:25.976798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:25.976798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:25.976798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:25.976798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:25.976798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:25.976798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:25.976798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:25.976798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:25.976798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:25.976798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:25.977942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:23:25.977942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:23:25.977942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:23:25.977942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:23:25.977942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:23:25.977942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:23:25.977942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:25.977942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:25.977942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 234/461] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:25.978419 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c019abca-3e7b-4132-bd4b-a87dbe4a03c2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:25.978488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930a2ad-aee4-4100-b51d-a41604694ba0 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:26.007275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:26.091454 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930a2ad-aee4-4100-b51d-a41604694ba0 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:26.092486 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c019abca-3e7b-4132-bd4b-a87dbe4a03c2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:26.115237 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:26.128078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ca01d-0951-4123-806c-e300370c88ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:26.128713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ca01d-0951-4123-806c-e300370c88ea None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:26.129030 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ca01d-0951-4123-806c-e300370c88ea None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:26.129573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ca01d-0951-4123-806c-e300370c88ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:26.130267 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f2cb6-57ed-49db-a55b-1c12fcaae90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:26.130875 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f2cb6-57ed-49db-a55b-1c12fcaae90c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:26.131159 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f2cb6-57ed-49db-a55b-1c12fcaae90c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:26.131436 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f2cb6-57ed-49db-a55b-1c12fcaae90c None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:26.132078 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f2cb6-57ed-49db-a55b-1c12fcaae90c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:26.132384 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f2cb6-57ed-49db-a55b-1c12fcaae90c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:26.133774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f2cb6-57ed-49db-a55b-1c12fcaae90c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:26.141284 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ca01d-0951-4123-806c-e300370c88ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:26.142753 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ca01d-0951-4123-806c-e300370c88ea None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:26.150662 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 228/462] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:26.150662 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f2cb6-57ed-49db-a55b-1c12fcaae90c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:26.160669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-201ca01d-0951-4123-806c-e300370c88ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:26.160669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:26.160669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:26.160669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:26.160669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:26.160669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:26.160669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:26.160669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:26.160669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:26.160669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:26.160669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:26.160669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:26.160669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:26.160669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:26.160669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:26.160669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:26.161823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:26.161823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:26.161823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:26.161823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:26.161823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:26.161823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:26.161823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:26.161823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:26.161823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:26.161823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:26.161823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:26.161823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:26.161823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:26.161823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:26.161823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:26.161823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:26.161823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:26.162854 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:26.162854 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:26.162854 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:26.168169 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 235/463] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:26.168169 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ca01d-0951-4123-806c-e300370c88ea None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:26.250980 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:26.262142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f2cb6-57ed-49db-a55b-1c12fcaae90c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:26.275413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ca01d-0951-4123-806c-e300370c88ea None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:26.299862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b790f75-331a-422e-a828-0dd1482e7b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:26.301567 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b790f75-331a-422e-a828-0dd1482e7b64 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:26.301846 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b790f75-331a-422e-a828-0dd1482e7b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:26.302115 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b790f75-331a-422e-a828-0dd1482e7b64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:26.308580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7e2b4-e2ce-4186-a74c-47e24f31a93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:26.309216 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7e2b4-e2ce-4186-a74c-47e24f31a93b None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:26.309495 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7e2b4-e2ce-4186-a74c-47e24f31a93b None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:26.309766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7e2b4-e2ce-4186-a74c-47e24f31a93b None admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:26.310453 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca7e2b4-e2ce-4186-a74c-47e24f31a93b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:26.310758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca7e2b4-e2ce-4186-a74c-47e24f31a93b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:26.312211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca7e2b4-e2ce-4186-a74c-47e24f31a93b None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:26.314164 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b790f75-331a-422e-a828-0dd1482e7b64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:26.315886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b790f75-331a-422e-a828-0dd1482e7b64 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:26.329329 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 236/464] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 20 13:23:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:26.329329 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7e2b4-e2ce-4186-a74c-47e24f31a93b None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:26.337778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-1b790f75-331a-422e-a828-0dd1482e7b64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:26.337778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:26.337778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:26.337778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:26.337778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:26.337778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:26.337778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:26.337778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:26.337778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:26.337778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:26.337778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:26.337778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:26.337778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:26.337778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:26.337778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:26.337778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:26.338886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:26.338886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:26.338886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:26.338886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:26.338886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:26.338886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:26.338886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:26.338886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:26.338886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:26.338886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:26.338886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:26.338886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:26.338886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:26.338886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:26.338886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:26.338886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:26.338886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:26.339908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:26.339908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:26.339908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:26.340198 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 229/465] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:26.341833 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b790f75-331a-422e-a828-0dd1482e7b64 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:26.377271 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:26.440824 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7e2b4-e2ce-4186-a74c-47e24f31a93b None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:26.455747 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b790f75-331a-422e-a828-0dd1482e7b64 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:26.475841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559afe2-92d3-4211-9bd5-56b66c055c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:26.476491 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559afe2-92d3-4211-9bd5-56b66c055c2d None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:26.476770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559afe2-92d3-4211-9bd5-56b66c055c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:26.477075 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559afe2-92d3-4211-9bd5-56b66c055c2d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:26.480333 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:26.486326 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559afe2-92d3-4211-9bd5-56b66c055c2d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:26.487892 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559afe2-92d3-4211-9bd5-56b66c055c2d None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:26.496880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0543a00-9a20-4f7d-9cc0-1b3e6c5b28fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:26.497494 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0543a00-9a20-4f7d-9cc0-1b3e6c5b28fa None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:26.497775 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0543a00-9a20-4f7d-9cc0-1b3e6c5b28fa None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:26.498047 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0543a00-9a20-4f7d-9cc0-1b3e6c5b28fa None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:26.498694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0543a00-9a20-4f7d-9cc0-1b3e6c5b28fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:26.498994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0543a00-9a20-4f7d-9cc0-1b3e6c5b28fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:26.500369 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0543a00-9a20-4f7d-9cc0-1b3e6c5b28fa None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:26.505890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-8559afe2-92d3-4211-9bd5-56b66c055c2d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 20 13:23:26.505890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:26.505890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:26.505890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:26.505890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:26.505890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:26.505890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:26.505890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:26.505890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:26.505890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:26.505890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:26.505890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:26.505890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:23:26.505890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:23:26.505890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:23:26.505890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:23:26.506781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:26.506781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:26.506781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:23:26.506781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:26.506781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:23:26.506781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:26.506781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:26.506781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:26.506781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:26.506781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:26.506781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:26.506781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:26.506781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:26.506781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:26.506781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:26.506781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:26.506781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:26.506781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:26.507669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:26.507669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:26.507669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:23:26.507669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:23:26.507669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:23:26.507669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:23:26.507669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:23:26.507669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:23:26.507669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 20 13:23:26.507669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:26.518625 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 237/466] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:23:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:26.518625 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559afe2-92d3-4211-9bd5-56b66c055c2d None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:26.518782 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 230/467] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:26.518782 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0543a00-9a20-4f7d-9cc0-1b3e6c5b28fa None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:26.615000 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:26.624277 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559afe2-92d3-4211-9bd5-56b66c055c2d None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:26.627423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0543a00-9a20-4f7d-9cc0-1b3e6c5b28fa None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:26.660829 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de993a2f-c930-4bd5-9102-661acae1867f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:26.661458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de993a2f-c930-4bd5-9102-661acae1867f None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:26.661734 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de993a2f-c930-4bd5-9102-661acae1867f None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:26.662002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de993a2f-c930-4bd5-9102-661acae1867f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:26.662278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1fe24-3e87-4b54-8c54-e79d6dea930f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:26.662775 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de993a2f-c930-4bd5-9102-661acae1867f None admin] RBAC: Adding query filter params (role.id=0f01a310371a4f6bbae7bb2e319a3b39, scope.project.id=4a81b6ecfb6e46a8b08c8bc2b4000fc7, user.id=ee5c2a3855cb4a668177aa19f337778d) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:26.662898 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1fe24-3e87-4b54-8c54-e79d6dea930f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:26.663082 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de993a2f-c930-4bd5-9102-661acae1867f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:26.663175 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1fe24-3e87-4b54-8c54-e79d6dea930f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:26.663386 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1fe24-3e87-4b54-8c54-e79d6dea930f None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:26.664016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1fe24-3e87-4b54-8c54-e79d6dea930f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:26.664316 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1fe24-3e87-4b54-8c54-e79d6dea930f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:26.664505 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de993a2f-c930-4bd5-9102-661acae1867f None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:26.665675 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1fe24-3e87-4b54-8c54-e79d6dea930f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:26.677752 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 231/468] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:26.682908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 238/469] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:23:26 2022] GET /identity/v3/role_assignments?user.id=ee5c2a3855cb4a668177aa19f337778d&scope.project.id=4a81b6ecfb6e46a8b08c8bc2b4000fc7&role.id=0f01a310371a4f6bbae7bb2e319a3b39 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:26.682988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1fe24-3e87-4b54-8c54-e79d6dea930f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:26.683106 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de993a2f-c930-4bd5-9102-661acae1867f None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:26.765513 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:26.812143 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1fe24-3e87-4b54-8c54-e79d6dea930f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:26.826234 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de993a2f-c930-4bd5-9102-661acae1867f None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:26.851715 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9d2fb-453f-4fd0-8f16-569b5759f7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:26.852357 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9d2fb-453f-4fd0-8f16-569b5759f7a9 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:26.852633 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9d2fb-453f-4fd0-8f16-569b5759f7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:26.853630 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9d2fb-453f-4fd0-8f16-569b5759f7a9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:26.864481 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9d2fb-453f-4fd0-8f16-569b5759f7a9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:26.865979 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9d2fb-453f-4fd0-8f16-569b5759f7a9 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:26.874108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707b506-781f-4f95-b8b3-7684016d49ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:26.875006 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707b506-781f-4f95-b8b3-7684016d49ac None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:26.875596 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707b506-781f-4f95-b8b3-7684016d49ac None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:26.875973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707b506-781f-4f95-b8b3-7684016d49ac None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:26.876797 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707b506-781f-4f95-b8b3-7684016d49ac None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:26.877231 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707b506-781f-4f95-b8b3-7684016d49ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:26.878716 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707b506-781f-4f95-b8b3-7684016d49ac None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:26.881717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-09b9d2fb-453f-4fd0-8f16-569b5759f7a9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 20 13:23:26.881717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:26.881717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:26.881717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:26.881717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:26.881717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:26.881717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:26.881717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:26.881717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:26.881717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:26.881717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:26.881717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:26.881717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:23:26.881717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:23:26.881717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:23:26.881717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:23:26.882750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:26.882750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:26.882750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:23:26.882750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:26.882750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:23:26.882750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:26.882750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:26.882750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:26.882750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:26.882750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:26.882750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:26.882750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:26.882750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:26.882750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:26.882750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:26.882750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:26.882750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:26.882750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:26.883814 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:26.883814 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:26.883814 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:23:26.883814 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:23:26.883814 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:23:26.883814 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:23:26.883814 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:23:26.883814 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:23:26.883814 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 20 13:23:26.883814 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:26.884309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 232/470] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 20 13:23:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:26.885011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9d2fb-453f-4fd0-8f16-569b5759f7a9 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:26.900820 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 239/471] 10.222.0.23 () {56 vars in 1137 bytes} [Thu Oct 20 13:23:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:26.901679 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707b506-781f-4f95-b8b3-7684016d49ac None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:26.915840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:27.022530 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9d2fb-453f-4fd0-8f16-569b5759f7a9 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:27.041715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707b506-781f-4f95-b8b3-7684016d49ac None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:27.060953 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:27.062197 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2ab89-1ebe-4b64-83c6-7a6d9ece18b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:27.062829 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2ab89-1ebe-4b64-83c6-7a6d9ece18b0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:27.063335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2ab89-1ebe-4b64-83c6-7a6d9ece18b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:27.063611 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2ab89-1ebe-4b64-83c6-7a6d9ece18b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:27.074044 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e2ab89-1ebe-4b64-83c6-7a6d9ece18b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:27.076309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e2ab89-1ebe-4b64-83c6-7a6d9ece18b0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:27.081418 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69372d66-0e52-41cd-8c3f-86bcf9944ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:27.082030 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69372d66-0e52-41cd-8c3f-86bcf9944ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:27.082308 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69372d66-0e52-41cd-8c3f-86bcf9944ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:27.082764 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69372d66-0e52-41cd-8c3f-86bcf9944ab3 None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:27.083567 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69372d66-0e52-41cd-8c3f-86bcf9944ab3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:27.083870 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69372d66-0e52-41cd-8c3f-86bcf9944ab3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:27.085639 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69372d66-0e52-41cd-8c3f-86bcf9944ab3 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:27.091634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-51e2ab89-1ebe-4b64-83c6-7a6d9ece18b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:27.091634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:27.091634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:27.091634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:27.091634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:27.091634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:27.091634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:27.091634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:27.091634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:27.091634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:27.091634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:27.091634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:27.091634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:23:27.091634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:23:27.091634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:23:27.092463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:23:27.092463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:27.092463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:27.092463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:27.092463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:27.092463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:27.092463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:27.092463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:27.092463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:27.092463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:27.092463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:27.092463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:27.092463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:27.092463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:27.092463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:27.092463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:27.092463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:27.093727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:23:27.093727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:23:27.093727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:23:27.093727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:23:27.093727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:23:27.093727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:23:27.093727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:27.093727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:27.096382 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 233/472] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:27.097212 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2ab89-1ebe-4b64-83c6-7a6d9ece18b0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:27.110687 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 240/473] 10.222.0.23 () {56 vars in 1139 bytes} [Thu Oct 20 13:23:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:27.111942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69372d66-0e52-41cd-8c3f-86bcf9944ab3 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:27.178081 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:27.215793 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2ab89-1ebe-4b64-83c6-7a6d9ece18b0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:27.224176 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69372d66-0e52-41cd-8c3f-86bcf9944ab3 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:27.258032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b725383-5b5a-487d-8505-1173819213e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:27.258781 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b725383-5b5a-487d-8505-1173819213e5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:27.259109 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b725383-5b5a-487d-8505-1173819213e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:27.259519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b725383-5b5a-487d-8505-1173819213e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:27.262144 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ebc9b-86b7-4b48-b58f-f61cb955c94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:27.263442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ebc9b-86b7-4b48-b58f-f61cb955c94d None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:27.263980 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ebc9b-86b7-4b48-b58f-f61cb955c94d None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:27.264439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ebc9b-86b7-4b48-b58f-f61cb955c94d None admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:27.265391 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ebc9b-86b7-4b48-b58f-f61cb955c94d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:27.265903 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ebc9b-86b7-4b48-b58f-f61cb955c94d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:27.267544 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ebc9b-86b7-4b48-b58f-f61cb955c94d None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:27.281005 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 241/474] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 20 13:23:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:27.282209 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ebc9b-86b7-4b48-b58f-f61cb955c94d None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:27.320733 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:27.340198 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b725383-5b5a-487d-8505-1173819213e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:27.342282 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b725383-5b5a-487d-8505-1173819213e5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:27.402055 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ebc9b-86b7-4b48-b58f-f61cb955c94d None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:27.431829 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:27.451451 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ed527e-31e7-4e19-96f4-7e46f8ecbd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:27.452618 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed527e-31e7-4e19-96f4-7e46f8ecbd8a None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:27.453088 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed527e-31e7-4e19-96f4-7e46f8ecbd8a None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:27.453492 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed527e-31e7-4e19-96f4-7e46f8ecbd8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:27.472799 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 234/475] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:23:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 13:23:27.472799 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b725383-5b5a-487d-8505-1173819213e5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:27.560520 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ed527e-31e7-4e19-96f4-7e46f8ecbd8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:27.562466 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ed527e-31e7-4e19-96f4-7e46f8ecbd8a None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:27.587206 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b725383-5b5a-487d-8505-1173819213e5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:27.608943 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:27.628298 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef152e3-ea34-427e-8a88-68e2eea30203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:27.629993 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef152e3-ea34-427e-8a88-68e2eea30203 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:27.631083 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef152e3-ea34-427e-8a88-68e2eea30203 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:27.631564 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef152e3-ea34-427e-8a88-68e2eea30203 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:27.632670 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef152e3-ea34-427e-8a88-68e2eea30203 None admin] RBAC: Adding query filter params (role.id=0f01a310371a4f6bbae7bb2e319a3b39, scope.project.id=4a81b6ecfb6e46a8b08c8bc2b4000fc7, user.id=87b5093e963c4753928a8c6091afc52b) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:27.633193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef152e3-ea34-427e-8a88-68e2eea30203 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:27.634941 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef152e3-ea34-427e-8a88-68e2eea30203 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:27.642497 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 235/476] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:23:27 2022] GET /identity/v3/role_assignments?user.id=87b5093e963c4753928a8c6091afc52b&scope.project.id=4a81b6ecfb6e46a8b08c8bc2b4000fc7&role.id=0f01a310371a4f6bbae7bb2e319a3b39 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:27.643617 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef152e3-ea34-427e-8a88-68e2eea30203 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:27.690019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:27.694897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 242/477] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:23:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 13:23:27.701393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ed527e-31e7-4e19-96f4-7e46f8ecbd8a None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:27.783772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef152e3-ea34-427e-8a88-68e2eea30203 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:27.826917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ed527e-31e7-4e19-96f4-7e46f8ecbd8a None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:27.828638 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142445e6-3461-4f0e-92c2-b0e7bdb08456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:27.829512 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142445e6-3461-4f0e-92c2-b0e7bdb08456 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:27.830014 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142445e6-3461-4f0e-92c2-b0e7bdb08456 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:27.830475 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142445e6-3461-4f0e-92c2-b0e7bdb08456 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:27.843697 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142445e6-3461-4f0e-92c2-b0e7bdb08456 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:27.845601 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142445e6-3461-4f0e-92c2-b0e7bdb08456 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:27.861537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-142445e6-3461-4f0e-92c2-b0e7bdb08456 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:27.861537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:27.861537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:27.861537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:27.861537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:27.861537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:27.861537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:27.861537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:27.861537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:27.861537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:27.861537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:27.861537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:27.861537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:27.861537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:27.861537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:27.861537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:27.862463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:27.862463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:27.862463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:27.862463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:27.862463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:27.862463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:27.862463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:27.862463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:27.862463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:27.862463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:27.862463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:27.862463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:27.862463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:27.862463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:27.862463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:27.862463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:27.862463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:27.863278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:27.863278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:27.863278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:27.865651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 236/478] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:27.871509 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142445e6-3461-4f0e-92c2-b0e7bdb08456 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:27.877127 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646ff60-2cf6-4646-a5f1-bfa21474020f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:27.877788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646ff60-2cf6-4646-a5f1-bfa21474020f None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:27.878063 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646ff60-2cf6-4646-a5f1-bfa21474020f None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:27.878330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646ff60-2cf6-4646-a5f1-bfa21474020f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:27.887308 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:27.912577 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646ff60-2cf6-4646-a5f1-bfa21474020f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:27.915520 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646ff60-2cf6-4646-a5f1-bfa21474020f None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:27.937573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-e646ff60-2cf6-4646-a5f1-bfa21474020f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:27.937573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:27.937573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:27.937573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:27.937573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:27.937573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:27.937573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:27.937573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:27.937573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:27.937573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:27.937573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:27.937573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:27.937573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:27.937573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:27.937573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:27.937573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:27.938487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:27.938487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:27.938487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:27.938487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:27.938487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:27.938487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:27.938487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:27.938487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:27.938487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:27.938487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:27.938487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:27.938487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:27.938487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:27.938487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:27.938487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:27.938487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:27.938487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:27.939303 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:27.939303 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:27.939303 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:27.941184 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 243/479] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:23:27.944964 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297d6f5-8135-4baa-90c3-6c684848a6b3 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:27.945229 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297d6f5-8135-4baa-90c3-6c684848a6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:27.947327 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297d6f5-8135-4baa-90c3-6c684848a6b3 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:27.947327 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 244/480] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:27.948006 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4297d6f5-8135-4baa-90c3-6c684848a6b3 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:28.011992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:28.025401 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142445e6-3461-4f0e-92c2-b0e7bdb08456 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:28.064567 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852be52-967b-4318-8e5b-d7a61123ece3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:28.065208 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852be52-967b-4318-8e5b-d7a61123ece3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:28.065487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852be52-967b-4318-8e5b-d7a61123ece3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:28.065767 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852be52-967b-4318-8e5b-d7a61123ece3 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:28.066416 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852be52-967b-4318-8e5b-d7a61123ece3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:28.066755 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852be52-967b-4318-8e5b-d7a61123ece3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:28.068243 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852be52-967b-4318-8e5b-d7a61123ece3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:28.077837 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4297d6f5-8135-4baa-90c3-6c684848a6b3 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:28.081293 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 237/481] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:28.083222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c1eb5-76cf-40c1-996f-fa03ed187b05 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:28.083513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c1eb5-76cf-40c1-996f-fa03ed187b05 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:28.083787 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c1eb5-76cf-40c1-996f-fa03ed187b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:28.102041 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0c4c1eb5-76cf-40c1-996f-fa03ed187b05 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:28.103194 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0c4c1eb5-76cf-40c1-996f-fa03ed187b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:28.124179 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6aabf-2d4d-4926-967e-2badd7090305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:28.125116 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6aabf-2d4d-4926-967e-2badd7090305 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:28.126061 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6aabf-2d4d-4926-967e-2badd7090305 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:28.126477 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6aabf-2d4d-4926-967e-2badd7090305 None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:28.127498 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6aabf-2d4d-4926-967e-2badd7090305 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:28.128203 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6aabf-2d4d-4926-967e-2badd7090305 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:28.130835 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6aabf-2d4d-4926-967e-2badd7090305 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:28.147878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 245/482] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:23:28.148924 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6aabf-2d4d-4926-967e-2badd7090305 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:28.181004 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 238/483] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:28.181917 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c1eb5-76cf-40c1-996f-fa03ed187b05 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:28.210530 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:28.273693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6aabf-2d4d-4926-967e-2badd7090305 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:28.314143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e832be-0bba-4d21-8405-a10cc577390f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:28.315001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e832be-0bba-4d21-8405-a10cc577390f None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:28.315417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e832be-0bba-4d21-8405-a10cc577390f None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:28.315851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e832be-0bba-4d21-8405-a10cc577390f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:28.325496 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e832be-0bba-4d21-8405-a10cc577390f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:28.327460 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e832be-0bba-4d21-8405-a10cc577390f None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:28.341988 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-98e832be-0bba-4d21-8405-a10cc577390f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:28.341988 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:28.341988 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:28.341988 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:28.341988 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:28.341988 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:28.341988 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:28.341988 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:28.341988 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:28.341988 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:28.341988 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:28.341988 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:28.341988 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:23:28.341988 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:23:28.341988 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:23:28.343062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:23:28.343062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:28.343062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:28.343062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:28.343062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:28.343062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:28.343062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:28.343062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:28.343062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:28.343062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:28.343062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:28.343062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:28.343062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:28.343062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:28.343062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:28.343062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:28.343062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:28.344068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:23:28.344068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:23:28.344068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:23:28.344068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:23:28.344068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:23:28.344068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:23:28.344068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:28.344068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:28.346468 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 246/484] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:28.351817 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e832be-0bba-4d21-8405-a10cc577390f None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:28.355858 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:28.390199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c1eb5-76cf-40c1-996f-fa03ed187b05 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:28.459541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021fadf8-ed04-406d-9231-b11b7325b4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:28.460411 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fadf8-ed04-406d-9231-b11b7325b4bc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:28.460829 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fadf8-ed04-406d-9231-b11b7325b4bc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:28.461282 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fadf8-ed04-406d-9231-b11b7325b4bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:28.475023 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e832be-0bba-4d21-8405-a10cc577390f None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:28.478035 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021fadf8-ed04-406d-9231-b11b7325b4bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:28.479717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021fadf8-ed04-406d-9231-b11b7325b4bc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:28.497266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:28.502477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-021fadf8-ed04-406d-9231-b11b7325b4bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:28.502477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:28.502477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:28.502477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:28.502477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:28.502477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:28.502477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:28.502477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:28.502477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:28.502477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:28.502477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:28.502477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:28.502477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:23:28.502477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:23:28.502477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:23:28.504001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:23:28.504001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:28.504001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:28.504001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:28.504001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:28.504001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:28.504001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:28.504001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:28.504001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:28.504001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:28.504001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:28.504001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:28.504001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:28.504001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:28.504001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:28.504001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:28.504001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:28.509007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:23:28.509007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:23:28.509007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:23:28.509007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:23:28.509007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:23:28.509007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:23:28.509007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:28.509007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:28.511561 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 239/485] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:28.517484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021fadf8-ed04-406d-9231-b11b7325b4bc None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:28.549381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af1b0e-1f38-4f44-a909-0f5f0c932759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:28.550910 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af1b0e-1f38-4f44-a909-0f5f0c932759 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:28.551390 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af1b0e-1f38-4f44-a909-0f5f0c932759 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:28.552047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af1b0e-1f38-4f44-a909-0f5f0c932759 None admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:28.553160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4af1b0e-1f38-4f44-a909-0f5f0c932759 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:28.553978 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4af1b0e-1f38-4f44-a909-0f5f0c932759 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:28.555853 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4af1b0e-1f38-4f44-a909-0f5f0c932759 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:28.572899 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 247/486] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 20 13:23:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:28.574354 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af1b0e-1f38-4f44-a909-0f5f0c932759 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:28.646352 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021fadf8-ed04-406d-9231-b11b7325b4bc None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:28.681859 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af1b0e-1f38-4f44-a909-0f5f0c932759 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:28.689553 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193bbac-752b-47c6-a114-e1f23b3666cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:28.690162 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193bbac-752b-47c6-a114-e1f23b3666cb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:28.690438 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193bbac-752b-47c6-a114-e1f23b3666cb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:28.690707 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193bbac-752b-47c6-a114-e1f23b3666cb None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:28.691463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193bbac-752b-47c6-a114-e1f23b3666cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:28.692288 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193bbac-752b-47c6-a114-e1f23b3666cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:28.693869 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193bbac-752b-47c6-a114-e1f23b3666cb None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:28.701338 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:28.711626 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 240/487] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 20 13:23:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:28.711626 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193bbac-752b-47c6-a114-e1f23b3666cb None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:28.730892 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370290cd-68b2-4b72-8ff9-c5d2173300a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:28.731703 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370290cd-68b2-4b72-8ff9-c5d2173300a6 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:28.732054 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370290cd-68b2-4b72-8ff9-c5d2173300a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:28.732497 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370290cd-68b2-4b72-8ff9-c5d2173300a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:28.743360 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370290cd-68b2-4b72-8ff9-c5d2173300a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:28.748283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-370290cd-68b2-4b72-8ff9-c5d2173300a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:23:28.748283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:28.748283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:28.748283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:28.748283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:28.748283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:28.748283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:28.748283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:28.748283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:28.748283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:28.748283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:28.748283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:28.748283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:28.748283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:28.748283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:28.748283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:28.749283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:28.749283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:28.749283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:28.749283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:28.749283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:28.749283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:28.749283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:28.749283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:28.749283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:28.749283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:28.749283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:28.749283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:28.749283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:28.749283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:28.749283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:28.749283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:28.750206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:28.750206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:28.750206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:28.750206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:28.750206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:28.750206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:28.750206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:28.750206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:28.750206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:28.750206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:28.750206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:23:28.750206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:28.757218 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 248/488] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 20 13:23:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:28.758183 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370290cd-68b2-4b72-8ff9-c5d2173300a6 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:28.788980 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:28.857410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193bbac-752b-47c6-a114-e1f23b3666cb None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:28.873063 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370290cd-68b2-4b72-8ff9-c5d2173300a6 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:28.907974 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1afdac-512e-4815-bb96-e094c2f327fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:28.908930 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1afdac-512e-4815-bb96-e094c2f327fa None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:28.909645 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1afdac-512e-4815-bb96-e094c2f327fa None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:28.910123 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1afdac-512e-4815-bb96-e094c2f327fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:28.911094 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1afdac-512e-4815-bb96-e094c2f327fa None admin] RBAC: Adding query filter params (role.id=0f01a310371a4f6bbae7bb2e319a3b39, scope.project.id=4a81b6ecfb6e46a8b08c8bc2b4000fc7, user.id=349ba417c3fc49ecaa11a65066dc54f8) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:28.911618 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1afdac-512e-4815-bb96-e094c2f327fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:28.913301 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1afdac-512e-4815-bb96-e094c2f327fa None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:28.914499 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f968d71-4fb9-4d1d-81a4-b26394a6654d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:28.915111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f968d71-4fb9-4d1d-81a4-b26394a6654d None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:28.915396 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f968d71-4fb9-4d1d-81a4-b26394a6654d None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:28.915670 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f968d71-4fb9-4d1d-81a4-b26394a6654d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:28.916400 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f968d71-4fb9-4d1d-81a4-b26394a6654d None admin] RBAC: Adding query filter params (role.id=0f01a310371a4f6bbae7bb2e319a3b39, scope.project.id=4a81b6ecfb6e46a8b08c8bc2b4000fc7, user.id=8a101adfc7a94bf1a4777350e9210a3a) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:28.916710 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f968d71-4fb9-4d1d-81a4-b26394a6654d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:28.918232 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f968d71-4fb9-4d1d-81a4-b26394a6654d None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:28.922470 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 241/489] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:23:28 2022] GET /identity/v3/role_assignments?user.id=349ba417c3fc49ecaa11a65066dc54f8&scope.project.id=4a81b6ecfb6e46a8b08c8bc2b4000fc7&role.id=0f01a310371a4f6bbae7bb2e319a3b39 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:28.923581 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1afdac-512e-4815-bb96-e094c2f327fa None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:28.931253 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 249/490] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:23:28 2022] GET /identity/v3/role_assignments?user.id=8a101adfc7a94bf1a4777350e9210a3a&scope.project.id=4a81b6ecfb6e46a8b08c8bc2b4000fc7&role.id=0f01a310371a4f6bbae7bb2e319a3b39 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:28.932591 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b83fe-8a5a-4c0b-979f-90daa933cd40 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:28.932897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b83fe-8a5a-4c0b-979f-90daa933cd40 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:28.933183 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b83fe-8a5a-4c0b-979f-90daa933cd40 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:28.934390 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 250/491] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:28.940102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f014be-d23b-41c6-9bf3-c5ec6243cbde None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:28.940416 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f014be-d23b-41c6-9bf3-c5ec6243cbde None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:28.940659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f014be-d23b-41c6-9bf3-c5ec6243cbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:28.986079 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-e7f014be-d23b-41c6-9bf3-c5ec6243cbde None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:28.987334 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-e7f014be-d23b-41c6-9bf3-c5ec6243cbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:29.006606 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:29.083687 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1afdac-512e-4815-bb96-e094c2f327fa None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:29.117197 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 251/492] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 13:23:29.171621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8b5e3-fa26-4009-9917-6f127d6506a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:29.172157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8b5e3-fa26-4009-9917-6f127d6506a5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:29.172439 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8b5e3-fa26-4009-9917-6f127d6506a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:29.172713 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8b5e3-fa26-4009-9917-6f127d6506a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:29.173429 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a8b5e3-fa26-4009-9917-6f127d6506a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:29.173738 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a8b5e3-fa26-4009-9917-6f127d6506a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:29.175112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a8b5e3-fa26-4009-9917-6f127d6506a5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:29.184466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 242/493] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 20 13:23:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:29.190197 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f014be-d23b-41c6-9bf3-c5ec6243cbde None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:29.218798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:29.301327 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f014be-d23b-41c6-9bf3-c5ec6243cbde None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:29.337579 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b657a-52c9-48cf-8a8b-5aad71b349c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:29.338266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b657a-52c9-48cf-8a8b-5aad71b349c9 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:29.338545 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b657a-52c9-48cf-8a8b-5aad71b349c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:29.338816 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b657a-52c9-48cf-8a8b-5aad71b349c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:29.349547 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b657a-52c9-48cf-8a8b-5aad71b349c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:29.350999 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b657a-52c9-48cf-8a8b-5aad71b349c9 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:29.365884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-a79b657a-52c9-48cf-8a8b-5aad71b349c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:29.365884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:29.365884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:29.365884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:29.365884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:29.365884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:29.365884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:29.365884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:29.365884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:29.365884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:29.365884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:29.365884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:29.365884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:29.365884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:29.365884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:29.365884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:29.367188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:29.367188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:29.367188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:29.367188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:29.367188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:29.367188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:29.367188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:29.367188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:29.367188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:29.367188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:29.367188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:29.367188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:29.367188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:29.367188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:29.367188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:29.367188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:29.367188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:29.369233 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:29.369233 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:29.369233 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:29.369233 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 252/494] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:29.372904 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8b5e3-fa26-4009-9917-6f127d6506a5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:29.437046 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b657a-52c9-48cf-8a8b-5aad71b349c9 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:29.545598 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8b5e3-fa26-4009-9917-6f127d6506a5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:29.574281 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:29.602013 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:29.602597 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b657a-52c9-48cf-8a8b-5aad71b349c9 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:29.621947 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a987c-790a-4283-9bd8-a1a2b89d9c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:29.622572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a987c-790a-4283-9bd8-a1a2b89d9c1e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:29.622857 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a987c-790a-4283-9bd8-a1a2b89d9c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:29.623135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a987c-790a-4283-9bd8-a1a2b89d9c1e None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:29.623891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a987c-790a-4283-9bd8-a1a2b89d9c1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:29.624205 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a987c-790a-4283-9bd8-a1a2b89d9c1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:29.634160 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a987c-790a-4283-9bd8-a1a2b89d9c1e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:29.652651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6050fef-3889-41ed-8eb7-43d0d8acd2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:29.653788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6050fef-3889-41ed-8eb7-43d0d8acd2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:29.656067 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6050fef-3889-41ed-8eb7-43d0d8acd2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:29.656495 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6050fef-3889-41ed-8eb7-43d0d8acd2a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:29.657459 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 243/495] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:29.665820 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a987c-790a-4283-9bd8-a1a2b89d9c1e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:29.675440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6050fef-3889-41ed-8eb7-43d0d8acd2a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:29.682897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-c6050fef-3889-41ed-8eb7-43d0d8acd2a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:23:29.682897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:29.682897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:29.682897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:29.682897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:29.682897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:29.682897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:29.682897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:29.682897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:29.682897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:29.682897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:29.682897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:29.682897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:29.682897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:29.682897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:29.682897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:29.683951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:29.683951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:29.683951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:29.683951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:29.683951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:29.683951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:29.683951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:29.683951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:29.683951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:29.683951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:29.683951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:29.683951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:29.683951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:29.683951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:29.683951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:29.683951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:29.685043 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:29.685043 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:29.685043 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:29.685043 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:29.685043 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:29.685043 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:29.685043 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:29.685043 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:29.685043 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:29.685043 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:29.685043 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:23:29.685043 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:29.687862 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 253/496] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 20 13:23:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:29.694411 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6050fef-3889-41ed-8eb7-43d0d8acd2a9 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:29.836510 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6050fef-3889-41ed-8eb7-43d0d8acd2a9 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:29.858884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a987c-790a-4283-9bd8-a1a2b89d9c1e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:29.877700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec837d-26f9-4fa3-a979-c870afb5cb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:29.878542 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec837d-26f9-4fa3-a979-c870afb5cb35 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:29.878984 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec837d-26f9-4fa3-a979-c870afb5cb35 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:29.880228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec837d-26f9-4fa3-a979-c870afb5cb35 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:29.881105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec837d-26f9-4fa3-a979-c870afb5cb35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:29.881551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec837d-26f9-4fa3-a979-c870afb5cb35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:29.883675 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec837d-26f9-4fa3-a979-c870afb5cb35 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:29.885797 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:29.896125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 254/497] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:23:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:29.901808 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec837d-26f9-4fa3-a979-c870afb5cb35 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:29.940428 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b6776-af3f-4c1b-b549-1b16ed3754a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:29.943312 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b6776-af3f-4c1b-b549-1b16ed3754a5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:29.945524 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b6776-af3f-4c1b-b549-1b16ed3754a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:29.945979 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b6776-af3f-4c1b-b549-1b16ed3754a5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:29.961493 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b6776-af3f-4c1b-b549-1b16ed3754a5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:29.964217 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b6776-af3f-4c1b-b549-1b16ed3754a5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:29.989305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-f99b6776-af3f-4c1b-b549-1b16ed3754a5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 20 13:23:29.989305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:29.989305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:29.989305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:29.989305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:29.989305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:29.989305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:29.989305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:29.989305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:29.989305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:29.989305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:29.989305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:29.989305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:23:29.989305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:23:29.989305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:23:29.989305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:23:29.990395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:29.990395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:29.990395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:23:29.990395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:29.990395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:23:29.990395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:29.990395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:29.990395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:29.990395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:29.990395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:29.990395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:29.990395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:29.990395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:29.990395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:29.990395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:29.990395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:29.990395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:29.990395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:29.991547 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:29.991547 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:29.991547 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:23:29.991547 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:23:29.991547 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:23:29.991547 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:23:29.991547 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:23:29.991547 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:23:29.991547 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 20 13:23:29.991547 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:29.991547 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 244/498] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 20 13:23:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:23:29.998128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b6776-af3f-4c1b-b549-1b16ed3754a5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:30.107079 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec837d-26f9-4fa3-a979-c870afb5cb35 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:30.108497 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:30.189752 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa351e3-9ac0-40c5-8e0c-8fd90053b62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:30.190397 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa351e3-9ac0-40c5-8e0c-8fd90053b62d None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:30.190732 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa351e3-9ac0-40c5-8e0c-8fd90053b62d None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:30.190947 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa351e3-9ac0-40c5-8e0c-8fd90053b62d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:30.205675 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b6776-af3f-4c1b-b549-1b16ed3754a5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:30.217649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa351e3-9ac0-40c5-8e0c-8fd90053b62d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:30.219120 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa351e3-9ac0-40c5-8e0c-8fd90053b62d None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:30.243348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ecdb4-5db9-4eac-a4e2-174223954860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:30.243580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-afa351e3-9ac0-40c5-8e0c-8fd90053b62d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:30.243580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:30.243580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:30.243580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:30.243580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:30.243580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:30.243580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:30.243580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:30.243580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:30.243580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:30.243580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:30.243580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:30.243580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:30.243580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:30.243580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:30.243580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:30.244432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:30.244432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:30.244432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:30.244432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:30.244432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:30.244432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:30.244432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:30.244432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:30.244432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:30.244432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:30.244432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:30.244432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:30.244432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:30.244432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:30.244432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:30.244432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:30.244432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:30.245838 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:30.245838 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:30.245838 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:30.247998 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ecdb4-5db9-4eac-a4e2-174223954860 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:30.248554 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ecdb4-5db9-4eac-a4e2-174223954860 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:30.249052 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ecdb4-5db9-4eac-a4e2-174223954860 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:30.249229 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 255/499] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:23:30.250268 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ecdb4-5db9-4eac-a4e2-174223954860 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:30.250778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ecdb4-5db9-4eac-a4e2-174223954860 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:30.252430 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ecdb4-5db9-4eac-a4e2-174223954860 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:30.253762 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa351e3-9ac0-40c5-8e0c-8fd90053b62d None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:30.275377 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 245/500] 10.222.0.23 () {56 vars in 1135 bytes} [Thu Oct 20 13:23:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:23:30.277130 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e1f61-fd08-4391-974a-13f86d92576c None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:30.277655 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e1f61-fd08-4391-974a-13f86d92576c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:30.278170 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e1f61-fd08-4391-974a-13f86d92576c None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:30.279823 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 246/501] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:30.281596 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e1f61-fd08-4391-974a-13f86d92576c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:30.321011 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:30.381668 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa351e3-9ac0-40c5-8e0c-8fd90053b62d None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:30.390988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e1f61-fd08-4391-974a-13f86d92576c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:30.393182 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:30.421945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c16c6-5a4b-4710-ac5d-cfe141af0933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:30.422413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c16c6-5a4b-4710-ac5d-cfe141af0933 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:30.422701 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c16c6-5a4b-4710-ac5d-cfe141af0933 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:30.422979 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c16c6-5a4b-4710-ac5d-cfe141af0933 None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:30.423639 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018c16c6-5a4b-4710-ac5d-cfe141af0933 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:30.423989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018c16c6-5a4b-4710-ac5d-cfe141af0933 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:30.425541 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018c16c6-5a4b-4710-ac5d-cfe141af0933 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:30.432239 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7141b-87ab-47d6-9538-91522519e38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:30.433141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7141b-87ab-47d6-9538-91522519e38d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:30.433651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7141b-87ab-47d6-9538-91522519e38d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:30.434139 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7141b-87ab-47d6-9538-91522519e38d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:30.439549 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 256/502] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:30.440931 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb64f6-972c-4ac1-b880-f05342d40675 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:30.441170 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb64f6-972c-4ac1-b880-f05342d40675 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:30.441448 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb64f6-972c-4ac1-b880-f05342d40675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:30.461285 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-b0fb64f6-972c-4ac1-b880-f05342d40675 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:30.462645 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b0fb64f6-972c-4ac1-b880-f05342d40675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:30.512308 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7141b-87ab-47d6-9538-91522519e38d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:30.514399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7141b-87ab-47d6-9538-91522519e38d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:30.538622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 257/503] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:30.539415 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb64f6-972c-4ac1-b880-f05342d40675 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:30.592270 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:30.623574 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 247/504] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:23:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 20 13:23:30.628910 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7141b-87ab-47d6-9538-91522519e38d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:30.632050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:30.709575 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb64f6-972c-4ac1-b880-f05342d40675 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:30.744488 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7141b-87ab-47d6-9538-91522519e38d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:30.744846 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac262f0c-95c9-4a04-8dc6-4998063f2537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:30.745527 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac262f0c-95c9-4a04-8dc6-4998063f2537 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:30.745812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac262f0c-95c9-4a04-8dc6-4998063f2537 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:30.746090 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac262f0c-95c9-4a04-8dc6-4998063f2537 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:30.768519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac262f0c-95c9-4a04-8dc6-4998063f2537 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:30.770386 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac262f0c-95c9-4a04-8dc6-4998063f2537 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:30.793266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-ac262f0c-95c9-4a04-8dc6-4998063f2537 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 20 13:23:30.793266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:30.793266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:30.793266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:30.793266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:30.793266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:30.793266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:30.793266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:30.793266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:30.793266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:30.793266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:30.793266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:30.793266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:23:30.793266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:23:30.793266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:23:30.793266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:23:30.794194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:30.794194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:30.794194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:23:30.794194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:30.794194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:23:30.794194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:30.794194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:30.794194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:30.794194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:30.794194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:30.794194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:30.794194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:30.794194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:30.794194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:30.794194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:30.794194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:30.794194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:30.794194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:30.795120 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:30.795120 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:30.795120 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:23:30.795120 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:23:30.795120 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:23:30.795120 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:23:30.795120 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:23:30.795120 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:23:30.795120 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 20 13:23:30.795120 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:30.798829 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075837e-c3a7-45f6-bbbb-99c7b4a06318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:30.799738 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075837e-c3a7-45f6-bbbb-99c7b4a06318 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:30.800218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075837e-c3a7-45f6-bbbb-99c7b4a06318 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:30.800723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075837e-c3a7-45f6-bbbb-99c7b4a06318 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:30.804757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 258/505] 10.222.0.23 () {56 vars in 1093 bytes} [Thu Oct 20 13:23:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:30.808831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac262f0c-95c9-4a04-8dc6-4998063f2537 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:30.818228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5075837e-c3a7-45f6-bbbb-99c7b4a06318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:30.820046 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5075837e-c3a7-45f6-bbbb-99c7b4a06318 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:30.839649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-5075837e-c3a7-45f6-bbbb-99c7b4a06318 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:30.839649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:30.839649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:30.839649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:30.839649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:30.839649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:30.839649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:30.839649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:30.839649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:30.839649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:30.839649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:30.839649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:30.839649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:30.839649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:30.839649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:30.839649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:30.840621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:30.840621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:30.840621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:30.840621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:30.840621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:30.840621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:30.840621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:30.840621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:30.840621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:30.840621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:30.840621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:30.840621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:30.840621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:30.840621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:30.840621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:30.840621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:30.840621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:30.841625 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:30.841625 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:30.841625 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:30.843302 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 248/506] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:30.843965 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075837e-c3a7-45f6-bbbb-99c7b4a06318 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:30.906839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:30.955671 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:30.958086 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075837e-c3a7-45f6-bbbb-99c7b4a06318 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:30.962324 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac262f0c-95c9-4a04-8dc6-4998063f2537 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:30.997222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c43c43-29b9-474f-94dd-1dff79d30d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:30.997844 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c43c43-29b9-474f-94dd-1dff79d30d42 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:30.998123 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c43c43-29b9-474f-94dd-1dff79d30d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:30.998394 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c43c43-29b9-474f-94dd-1dff79d30d42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:31.013043 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c43c43-29b9-474f-94dd-1dff79d30d42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:31.014494 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c43c43-29b9-474f-94dd-1dff79d30d42 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:31.015725 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765794d-570b-4444-9ad8-39f38ff8050b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:31.017400 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765794d-570b-4444-9ad8-39f38ff8050b None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:31.017851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765794d-570b-4444-9ad8-39f38ff8050b None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:31.018270 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765794d-570b-4444-9ad8-39f38ff8050b None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:31.019151 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765794d-570b-4444-9ad8-39f38ff8050b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:31.019616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765794d-570b-4444-9ad8-39f38ff8050b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:31.021987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765794d-570b-4444-9ad8-39f38ff8050b None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:31.029410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-a7c43c43-29b9-474f-94dd-1dff79d30d42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:31.029410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:31.029410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:31.029410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:31.029410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:31.029410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:31.029410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:31.029410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:31.029410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:31.029410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:31.029410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:31.029410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:31.029410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:31.029410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:31.029410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:31.029410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:31.030521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:31.030521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:31.030521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:31.030521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:31.030521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:31.030521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:31.030521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:31.030521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:31.030521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:31.030521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:31.030521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:31.030521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:31.030521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:31.030521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:31.030521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:31.030521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:31.030521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:31.031514 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:31.031514 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:31.031514 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:31.031514 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 249/507] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:31.031514 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c43c43-29b9-474f-94dd-1dff79d30d42 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:31.046468 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 259/508] 10.222.0.23 () {56 vars in 1133 bytes} [Thu Oct 20 13:23:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:31.047613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765794d-570b-4444-9ad8-39f38ff8050b None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:31.156092 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c43c43-29b9-474f-94dd-1dff79d30d42 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:31.163353 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:31.164457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765794d-570b-4444-9ad8-39f38ff8050b None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:31.193503 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c54c67-b8c3-4a22-95ac-f1b88adfcdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:31.194112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c54c67-b8c3-4a22-95ac-f1b88adfcdfb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:31.194566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c54c67-b8c3-4a22-95ac-f1b88adfcdfb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:31.194683 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c54c67-b8c3-4a22-95ac-f1b88adfcdfb None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:31.195279 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c54c67-b8c3-4a22-95ac-f1b88adfcdfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:31.195646 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c54c67-b8c3-4a22-95ac-f1b88adfcdfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:31.197045 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c54c67-b8c3-4a22-95ac-f1b88adfcdfb None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:31.202937 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4fe5c-d714-4a3e-8bc7-078e3e0b6f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:31.203726 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4fe5c-d714-4a3e-8bc7-078e3e0b6f4f None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:31.204174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4fe5c-d714-4a3e-8bc7-078e3e0b6f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:31.204608 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4fe5c-d714-4a3e-8bc7-078e3e0b6f4f None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:31.205438 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4fe5c-d714-4a3e-8bc7-078e3e0b6f4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:31.205890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4fe5c-d714-4a3e-8bc7-078e3e0b6f4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:31.207958 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4fe5c-d714-4a3e-8bc7-078e3e0b6f4f None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:31.210819 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 250/509] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:31.211469 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c54c67-b8c3-4a22-95ac-f1b88adfcdfb None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:31.220856 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 260/510] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:31.221904 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4fe5c-d714-4a3e-8bc7-078e3e0b6f4f None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:31.242227 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:31.332181 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c54c67-b8c3-4a22-95ac-f1b88adfcdfb None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:31.335524 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4fe5c-d714-4a3e-8bc7-078e3e0b6f4f None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:31.372149 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c8903-da60-41d6-ac74-a629615bd0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:31.372811 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c8903-da60-41d6-ac74-a629615bd0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:31.373122 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c8903-da60-41d6-ac74-a629615bd0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:31.373362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96847473-6406-4f29-ac99-016ef0358c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:31.373512 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c8903-da60-41d6-ac74-a629615bd0c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:31.374471 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96847473-6406-4f29-ac99-016ef0358c3e None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:31.374910 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96847473-6406-4f29-ac99-016ef0358c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:31.375342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96847473-6406-4f29-ac99-016ef0358c3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:31.383425 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96847473-6406-4f29-ac99-016ef0358c3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:31.385203 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96847473-6406-4f29-ac99-016ef0358c3e None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:31.386053 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c8903-da60-41d6-ac74-a629615bd0c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:31.387915 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c8903-da60-41d6-ac74-a629615bd0c0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:31.399276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-96847473-6406-4f29-ac99-016ef0358c3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:31.399276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:31.399276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:31.399276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:31.399276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:31.399276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:31.399276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:31.399276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:31.399276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:31.399276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:31.399276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:31.399276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:31.399276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:23:31.399276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:23:31.399276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:23:31.400371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:23:31.400371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:31.400371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:31.400371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:31.400371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:31.400371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:31.400371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:31.400371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:31.400371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:31.400371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:31.400371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:31.400371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:31.400371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:31.400371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:31.400371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:31.400371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:31.400371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:31.401365 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:23:31.401365 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:23:31.401365 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:23:31.401365 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:23:31.401365 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:23:31.401365 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:23:31.401365 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:31.401365 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:31.403292 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 261/511] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:31.404171 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96847473-6406-4f29-ac99-016ef0358c3e None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:31.404733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-9d3c8903-da60-41d6-ac74-a629615bd0c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:31.404733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:31.404733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:31.404733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:31.404733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:31.404733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:31.404733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:31.404733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:31.404733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:31.404733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:31.404733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:31.404733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:31.404733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:31.404733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:31.404733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:31.404733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:31.405956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:31.405956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:31.405956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:31.405956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:31.405956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:31.405956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:31.405956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:31.405956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:31.405956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:31.405956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:31.405956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:31.405956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:31.405956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:31.405956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:31.405956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:31.405956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:31.405956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:31.406985 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:31.406985 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:31.406985 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:31.406985 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 251/512] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:31.413654 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c8903-da60-41d6-ac74-a629615bd0c0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:31.445284 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:31.492429 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:31.520978 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96847473-6406-4f29-ac99-016ef0358c3e None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:31.538563 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c8903-da60-41d6-ac74-a629615bd0c0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:31.556988 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a3251-7dcb-450b-8bc5-7a77c1b6ada5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:31.557595 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a3251-7dcb-450b-8bc5-7a77c1b6ada5 None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:31.557871 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a3251-7dcb-450b-8bc5-7a77c1b6ada5 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:31.558141 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a3251-7dcb-450b-8bc5-7a77c1b6ada5 None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:31.558776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13a3251-7dcb-450b-8bc5-7a77c1b6ada5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:31.560210 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13a3251-7dcb-450b-8bc5-7a77c1b6ada5 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:31.578865 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f54e78-a0c4-4f9d-9ab5-9667e2369c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:31.579954 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f54e78-a0c4-4f9d-9ab5-9667e2369c62 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:31.580409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f54e78-a0c4-4f9d-9ab5-9667e2369c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:31.580833 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f54e78-a0c4-4f9d-9ab5-9667e2369c62 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:31.581657 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f54e78-a0c4-4f9d-9ab5-9667e2369c62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:31.582108 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f54e78-a0c4-4f9d-9ab5-9667e2369c62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:31.583455 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 262/513] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 20 13:23:31 2022] POST /identity/v3/users => generated 254 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:23:31.584393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f54e78-a0c4-4f9d-9ab5-9667e2369c62 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:31.584580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a3251-7dcb-450b-8bc5-7a77c1b6ada5 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:31.604148 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 252/514] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:31.605820 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6df09d-8cb7-49bc-9590-480b20ff4987 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:31.606267 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6df09d-8cb7-49bc-9590-480b20ff4987 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:31.606690 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6df09d-8cb7-49bc-9590-480b20ff4987 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:31.608492 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 253/515] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:31.610325 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3efb4-bb2c-4012-95cb-ef001c38d504 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:31.610770 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3efb4-bb2c-4012-95cb-ef001c38d504 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:31.611769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3efb4-bb2c-4012-95cb-ef001c38d504 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:31.613570 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 254/516] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:31.614667 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3efb4-bb2c-4012-95cb-ef001c38d504 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:31.705175 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:31.709581 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a3251-7dcb-450b-8bc5-7a77c1b6ada5 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:31.732344 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3efb4-bb2c-4012-95cb-ef001c38d504 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:31.744721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:31.751676 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42efe7f2-da0c-4913-ab9a-f6d71a865d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:31.752844 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efe7f2-da0c-4913-ab9a-f6d71a865d09 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:31.753146 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efe7f2-da0c-4913-ab9a-f6d71a865d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:31.753427 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efe7f2-da0c-4913-ab9a-f6d71a865d09 None admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:31.754653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42efe7f2-da0c-4913-ab9a-f6d71a865d09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:31.755601 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42efe7f2-da0c-4913-ab9a-f6d71a865d09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:31.757098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42efe7f2-da0c-4913-ab9a-f6d71a865d09 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:31.770004 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 263/517] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 20 13:23:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:31.771379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb697c4f-7aea-4394-b2fd-19e3e9e46f4e None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:31.771670 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb697c4f-7aea-4394-b2fd-19e3e9e46f4e None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:31.771940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb697c4f-7aea-4394-b2fd-19e3e9e46f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:31.774237 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505dee7a-b24e-4ab3-91e9-40586742b35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:31.775097 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505dee7a-b24e-4ab3-91e9-40586742b35b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:31.775533 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505dee7a-b24e-4ab3-91e9-40586742b35b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:31.776003 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505dee7a-b24e-4ab3-91e9-40586742b35b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:31.783216 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505dee7a-b24e-4ab3-91e9-40586742b35b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:31.784727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505dee7a-b24e-4ab3-91e9-40586742b35b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:31.790810 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-eb697c4f-7aea-4394-b2fd-19e3e9e46f4e None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:31.791999 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-eb697c4f-7aea-4394-b2fd-19e3e9e46f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:31.799678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-505dee7a-b24e-4ab3-91e9-40586742b35b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:31.799678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:31.799678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:31.799678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:31.799678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:31.799678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:31.799678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:31.799678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:31.799678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:31.799678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:31.799678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:31.799678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:31.799678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:23:31.799678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:23:31.799678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:23:31.800551 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:23:31.800551 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:31.800551 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:31.800551 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:31.800551 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:31.800551 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:31.800551 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:31.800551 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:31.800551 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:31.800551 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:31.800551 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:31.800551 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:31.800551 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:31.800551 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:31.800551 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:31.800551 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:31.800551 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:31.803145 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:23:31.803145 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:23:31.803145 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:23:31.803145 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:23:31.803145 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:23:31.803145 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:23:31.803145 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:31.803145 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:31.803145 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 255/518] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:31.803145 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70a27e-9dfa-4c8b-8c71-ffe0b71e418e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:31.803145 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70a27e-9dfa-4c8b-8c71-ffe0b71e418e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:31.803744 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e70a27e-9dfa-4c8b-8c71-ffe0b71e418e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:31.824251 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2e70a27e-9dfa-4c8b-8c71-ffe0b71e418e None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:31.825463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2e70a27e-9dfa-4c8b-8c71-ffe0b71e418e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:31.884897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 264/519] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:31.884897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb697c4f-7aea-4394-b2fd-19e3e9e46f4e None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:31.913235 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 256/520] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:31.914310 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70a27e-9dfa-4c8b-8c71-ffe0b71e418e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:31.992229 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb697c4f-7aea-4394-b2fd-19e3e9e46f4e None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:32.018088 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:32.031394 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb26ce-7c9d-46d2-9373-8aa3bcce7550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:32.032073 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb26ce-7c9d-46d2-9373-8aa3bcce7550 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:32.032416 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb26ce-7c9d-46d2-9373-8aa3bcce7550 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:32.032929 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb26ce-7c9d-46d2-9373-8aa3bcce7550 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:32.034080 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb26ce-7c9d-46d2-9373-8aa3bcce7550 None admin] RBAC: Adding query filter params (role.id=0f01a310371a4f6bbae7bb2e319a3b39, scope.project.id=4a81b6ecfb6e46a8b08c8bc2b4000fc7, user.id=9834211488354651803000c7e8fab80b) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:32.034507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb26ce-7c9d-46d2-9373-8aa3bcce7550 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:32.036230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb26ce-7c9d-46d2-9373-8aa3bcce7550 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:32.049269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:32.049771 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 265/521] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:23:31 2022] GET /identity/v3/role_assignments?user.id=9834211488354651803000c7e8fab80b&scope.project.id=4a81b6ecfb6e46a8b08c8bc2b4000fc7&role.id=0f01a310371a4f6bbae7bb2e319a3b39 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:32.119220 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e70a27e-9dfa-4c8b-8c71-ffe0b71e418e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:32.161972 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb26ce-7c9d-46d2-9373-8aa3bcce7550 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:32.187142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e639cc-4752-42b7-8591-1c2d04541dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:32.187775 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e639cc-4752-42b7-8591-1c2d04541dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:32.188056 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e639cc-4752-42b7-8591-1c2d04541dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:32.188330 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e639cc-4752-42b7-8591-1c2d04541dd2 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:32.189193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e639cc-4752-42b7-8591-1c2d04541dd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:32.189507 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e639cc-4752-42b7-8591-1c2d04541dd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:32.191101 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e639cc-4752-42b7-8591-1c2d04541dd2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:32.215779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 257/522] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 20 13:23:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:32.221085 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e639cc-4752-42b7-8591-1c2d04541dd2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:32.310100 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb26ce-7c9d-46d2-9373-8aa3bcce7550 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:32.354105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa742a-8d6e-4448-8a36-0f67a1a0d25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:32.354841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa742a-8d6e-4448-8a36-0f67a1a0d25b None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:32.355440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa742a-8d6e-4448-8a36-0f67a1a0d25b None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:32.355721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa742a-8d6e-4448-8a36-0f67a1a0d25b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:32.356456 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa742a-8d6e-4448-8a36-0f67a1a0d25b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:32.358177 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa742a-8d6e-4448-8a36-0f67a1a0d25b None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:32.359801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e639cc-4752-42b7-8591-1c2d04541dd2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:32.373247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-9efa742a-8d6e-4448-8a36-0f67a1a0d25b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 13:23:32.373247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:32.373247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:23:32.373247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:23:32.373247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:23:32.373247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:23:32.373247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:23:32.373247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:32.373247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 13:23:32.373247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:32.373247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:23:32.373247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:32.373247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:32.373247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:32.373247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:32.373247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:32.373247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:32.374507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:32.374507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:32.374507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:32.374507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:32.374507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:32.374507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:32.374507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:23:32.374507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:23:32.374507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:23:32.374507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:23:32.374507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:32.374507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:32.374507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:32.374507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:32.374507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:32.374507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:32.375538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:32.375538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:32.375538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:32.375538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:32.375538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:32.375538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:32.375538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:32.375538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:32.375538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:32.375538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:32.375538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:23:32.375538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:32.375538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 13:23:32.375538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:32.378792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 266/523] 10.222.0.23 () {56 vars in 1129 bytes} [Thu Oct 20 13:23:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:32.378792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa742a-8d6e-4448-8a36-0f67a1a0d25b None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:32.390605 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:32.398484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded799e-48b7-4161-bfc8-966f8e18081b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:32.399398 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded799e-48b7-4161-bfc8-966f8e18081b None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:32.399869 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded799e-48b7-4161-bfc8-966f8e18081b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:32.400311 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded799e-48b7-4161-bfc8-966f8e18081b None admin] PATH_INFO: `/v3/projects/4a81b6ecfb6e46a8b08c8bc2b4000fc7/users/ee5c2a3855cb4a668177aa19f337778d/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:32.402582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded799e-48b7-4161-bfc8-966f8e18081b None admin] RBAC: Authorizing `identity:create_grant(project_id=4a81b6ecfb6e46a8b08c8bc2b4000fc7, user_id=ee5c2a3855cb4a668177aa19f337778d, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:32.404144 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded799e-48b7-4161-bfc8-966f8e18081b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:32.421432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 258/524] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:23:32 2022] PUT /identity/v3/projects/4a81b6ecfb6e46a8b08c8bc2b4000fc7/users/ee5c2a3855cb4a668177aa19f337778d/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:23:32.422278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded799e-48b7-4161-bfc8-966f8e18081b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:32.471049 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:32.559119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa742a-8d6e-4448-8a36-0f67a1a0d25b None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:32.560586 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded799e-48b7-4161-bfc8-966f8e18081b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:32.602251 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e98d7-06f0-4fce-963c-962010b704ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:32.603583 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e98d7-06f0-4fce-963c-962010b704ee None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:32.604217 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e98d7-06f0-4fce-963c-962010b704ee None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:32.604714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e98d7-06f0-4fce-963c-962010b704ee None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:32.605543 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633e98d7-06f0-4fce-963c-962010b704ee None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:32.605984 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633e98d7-06f0-4fce-963c-962010b704ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:32.607519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633e98d7-06f0-4fce-963c-962010b704ee None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:32.613260 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 259/525] 10.222.0.23 () {56 vars in 1124 bytes} [Thu Oct 20 13:23:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:32.619827 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e98d7-06f0-4fce-963c-962010b704ee None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:32.644339 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b6c63-9cc3-461d-80e0-658fc77177ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:32.645844 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b6c63-9cc3-461d-80e0-658fc77177ef None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:32.647767 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b6c63-9cc3-461d-80e0-658fc77177ef None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:32.648227 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b6c63-9cc3-461d-80e0-658fc77177ef None admin] PATH_INFO: `/v3/services/network` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:32.649039 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b6c63-9cc3-461d-80e0-658fc77177ef None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:32.651302 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93b6c63-9cc3-461d-80e0-658fc77177ef None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:32.663393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-d93b6c63-9cc3-461d-80e0-658fc77177ef None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 13:23:32.663393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:32.663393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:23:32.663393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:23:32.663393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:23:32.663393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:23:32.663393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:23:32.663393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:32.663393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 13:23:32.663393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:32.663393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:23:32.663393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:32.663393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:32.663393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:32.663393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:32.663393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:32.663393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:32.664543 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:32.664543 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:32.664543 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:32.664543 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:32.664543 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:32.664543 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:32.664543 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:23:32.664543 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:23:32.664543 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:23:32.664543 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:23:32.664543 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:32.664543 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:32.664543 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:32.664543 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:32.664543 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:32.664543 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:32.665413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:32.665413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:32.665413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:32.665413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:32.665413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:32.665413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:32.665413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:32.665413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:32.665413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:32.665413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:32.665413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:23:32.665413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:32.665413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 13:23:32.665413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:32.668051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 267/526] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:32 2022] GET /identity/v3/services/network => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:32.674903 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b6c63-9cc3-461d-80e0-658fc77177ef None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:32.726406 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:32.782905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e98d7-06f0-4fce-963c-962010b704ee None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:32.820046 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b6c63-9cc3-461d-80e0-658fc77177ef None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:32.838011 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:32.853673 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db06fe-9ce1-40d5-a0ae-05358241c7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:32.854302 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db06fe-9ce1-40d5-a0ae-05358241c7c5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:32.854589 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db06fe-9ce1-40d5-a0ae-05358241c7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:32.854867 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db06fe-9ce1-40d5-a0ae-05358241c7c5 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:32.855566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db06fe-9ce1-40d5-a0ae-05358241c7c5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:32.855876 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db06fe-9ce1-40d5-a0ae-05358241c7c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:32.857355 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db06fe-9ce1-40d5-a0ae-05358241c7c5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:32.861961 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26a6ff-595d-4495-83d5-622e1a877051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:32.862736 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26a6ff-595d-4495-83d5-622e1a877051 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:32.863184 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26a6ff-595d-4495-83d5-622e1a877051 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:32.864133 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26a6ff-595d-4495-83d5-622e1a877051 None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:32.865139 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26a6ff-595d-4495-83d5-622e1a877051 None admin] RBAC: Adding query filter params (name=network) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:32.865589 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26a6ff-595d-4495-83d5-622e1a877051 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:32.867565 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26a6ff-595d-4495-83d5-622e1a877051 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:32.870094 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 260/527] 10.222.0.23 () {56 vars in 1124 bytes} [Thu Oct 20 13:23:32 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:32.873069 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 268/528] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 20 13:23:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:23:32.882266 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db06fe-9ce1-40d5-a0ae-05358241c7c5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:33.038369 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db06fe-9ce1-40d5-a0ae-05358241c7c5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:33.058119 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:33.078234 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eef168-2b38-4541-9cdf-a7f640bb48eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:33.078870 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eef168-2b38-4541-9cdf-a7f640bb48eb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:33.079302 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eef168-2b38-4541-9cdf-a7f640bb48eb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:33.079648 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eef168-2b38-4541-9cdf-a7f640bb48eb None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:33.080361 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eef168-2b38-4541-9cdf-a7f640bb48eb None admin] RBAC: Adding query filter params (type=network) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:33.080785 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eef168-2b38-4541-9cdf-a7f640bb48eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:33.082942 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eef168-2b38-4541-9cdf-a7f640bb48eb None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:33.088846 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 261/529] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 20 13:23:32 2022] GET /identity/v3/services?type=network => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:34.438306 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f4fed-e175-4033-91ec-5a589b8d9db4 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:34.438840 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f4fed-e175-4033-91ec-5a589b8d9db4 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:34.438840 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f4fed-e175-4033-91ec-5a589b8d9db4 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:34.440434 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 269/530] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:34.446077 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c009a1-d2f1-4e58-9779-d45cca4fe504 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:34.446485 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c009a1-d2f1-4e58-9779-d45cca4fe504 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:34.446704 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c009a1-d2f1-4e58-9779-d45cca4fe504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:34.474093 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-70c009a1-d2f1-4e58-9779-d45cca4fe504 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:34.475308 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-70c009a1-d2f1-4e58-9779-d45cca4fe504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:34.569295 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 262/531] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 13:23:34.659397 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a917e3-68f9-4cc9-af57-aaf73fe0e262 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:34.659747 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a917e3-68f9-4cc9-af57-aaf73fe0e262 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:34.659937 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a917e3-68f9-4cc9-af57-aaf73fe0e262 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:34.661800 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 270/532] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:34.667152 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc7628-b5c5-493c-ab2c-94cfa78eeeda None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:34.667434 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc7628-b5c5-493c-ab2c-94cfa78eeeda None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:34.667705 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc7628-b5c5-493c-ab2c-94cfa78eeeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:34.689095 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f6dc7628-b5c5-493c-ab2c-94cfa78eeeda None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:34.690304 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f6dc7628-b5c5-493c-ab2c-94cfa78eeeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:34.780362 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 263/533] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:34.915949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a917e3-68f9-4cc9-af57-aaf73fe0e262 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:35.070651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a917e3-68f9-4cc9-af57-aaf73fe0e262 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:35.075643 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c3ac2-39a2-4784-9e16-5c59c051d52a None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:35.075955 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c3ac2-39a2-4784-9e16-5c59c051d52a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:35.076232 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c3ac2-39a2-4784-9e16-5c59c051d52a None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:35.078950 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 264/534] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:35.083769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c313b8d-5cb8-4455-9f07-7cc6b0b0e58a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:35.084060 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c313b8d-5cb8-4455-9f07-7cc6b0b0e58a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:35.084333 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c313b8d-5cb8-4455-9f07-7cc6b0b0e58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:35.111242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8c313b8d-5cb8-4455-9f07-7cc6b0b0e58a None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:35.112418 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8c313b8d-5cb8-4455-9f07-7cc6b0b0e58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:35.135162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a4b4f-953d-4f89-b4aa-ff48f99d9500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:35.135785 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a4b4f-953d-4f89-b4aa-ff48f99d9500 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:35.136059 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a4b4f-953d-4f89-b4aa-ff48f99d9500 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:35.136327 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a4b4f-953d-4f89-b4aa-ff48f99d9500 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:35.154535 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a4b4f-953d-4f89-b4aa-ff48f99d9500 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:35.161457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-551a4b4f-953d-4f89-b4aa-ff48f99d9500 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:23:35.161457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:35.161457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:35.161457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:35.161457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:35.161457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:35.161457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:35.161457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:35.161457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:35.161457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:35.161457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:35.161457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:35.161457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:35.161457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:35.161457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:35.161457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:35.162373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:35.162373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:35.162373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:35.162373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:35.162373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:35.162373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:35.162373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:35.162373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:35.162373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:35.162373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:35.162373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:35.162373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:35.162373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:35.162373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:35.162373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:35.162373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:35.163221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:35.163221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:35.163221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:35.163221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:35.163221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:35.163221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:35.163221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:35.163221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:35.163221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:35.163221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:35.163221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:23:35.163221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:35.166027 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 271/535] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 20 13:23:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:35.166551 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:35.168270 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a4b4f-953d-4f89-b4aa-ff48f99d9500 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:35.205020 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 265/536] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 20 13:23:35.205945 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c313b8d-5cb8-4455-9f07-7cc6b0b0e58a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:35.214213 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:35.281581 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a4b4f-953d-4f89-b4aa-ff48f99d9500 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:35.336161 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cabe202-7910-4ea8-9e7e-ca20cde8806e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:35.337055 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabe202-7910-4ea8-9e7e-ca20cde8806e None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:35.337477 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabe202-7910-4ea8-9e7e-ca20cde8806e None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:35.337910 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabe202-7910-4ea8-9e7e-ca20cde8806e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:35.350973 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c313b8d-5cb8-4455-9f07-7cc6b0b0e58a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:35.357493 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cabe202-7910-4ea8-9e7e-ca20cde8806e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:35.369796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-2cabe202-7910-4ea8-9e7e-ca20cde8806e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:23:35.369796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:35.369796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:35.369796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:35.369796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:35.369796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:35.369796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:35.369796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:35.369796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:35.369796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:35.369796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:35.369796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:35.369796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:35.369796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:35.369796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:35.369796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:35.370948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:35.370948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:35.370948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:35.370948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:35.370948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:35.370948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:35.370948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:35.370948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:35.370948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:35.370948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:35.370948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:35.370948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:35.370948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:35.370948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:35.370948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:35.370948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:35.371894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:35.371894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:35.371894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:35.371894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:35.371894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:35.371894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:35.371894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:35.371894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:35.371894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:35.371894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:35.371894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:23:35.371894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:35.378241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 272/537] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 20 13:23:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:35.384074 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cabe202-7910-4ea8-9e7e-ca20cde8806e None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:35.387052 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d7a84-114f-4103-a23f-5e934835a77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:35.387912 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d7a84-114f-4103-a23f-5e934835a77d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:35.388249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d7a84-114f-4103-a23f-5e934835a77d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:35.388522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d7a84-114f-4103-a23f-5e934835a77d None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:35.389210 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d7a84-114f-4103-a23f-5e934835a77d None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:35.389508 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d7a84-114f-4103-a23f-5e934835a77d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:35.390967 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d7a84-114f-4103-a23f-5e934835a77d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:35.405288 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 266/538] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 20 13:23:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:35.411127 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d7a84-114f-4103-a23f-5e934835a77d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:35.490538 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:35.603650 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:35.607870 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d7a84-114f-4103-a23f-5e934835a77d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:35.612028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cabe202-7910-4ea8-9e7e-ca20cde8806e None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:35.651362 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b49421-dd59-4f17-bede-8ad210d56a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:35.651992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b49421-dd59-4f17-bede-8ad210d56a19 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:35.652268 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b49421-dd59-4f17-bede-8ad210d56a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:35.652536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b49421-dd59-4f17-bede-8ad210d56a19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:35.658060 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279252e-0809-4d8e-93a6-76264ab3f27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:35.658662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279252e-0809-4d8e-93a6-76264ab3f27a None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:35.658934 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279252e-0809-4d8e-93a6-76264ab3f27a None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:35.659199 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279252e-0809-4d8e-93a6-76264ab3f27a None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:35.659857 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279252e-0809-4d8e-93a6-76264ab3f27a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:35.660666 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279252e-0809-4d8e-93a6-76264ab3f27a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:35.662554 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279252e-0809-4d8e-93a6-76264ab3f27a None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:35.666789 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b49421-dd59-4f17-bede-8ad210d56a19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:35.669035 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b49421-dd59-4f17-bede-8ad210d56a19 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:35.672060 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 273/539] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:23:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:23:35.672975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279252e-0809-4d8e-93a6-76264ab3f27a None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:35.687193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-07b49421-dd59-4f17-bede-8ad210d56a19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:35.687193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:35.687193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:35.687193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:35.687193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:35.687193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:35.687193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:35.687193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:35.687193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:35.687193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:35.687193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:35.687193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:35.687193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:35.687193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:35.687193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:35.687193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:35.688176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:35.688176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:35.688176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:35.688176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:35.688176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:35.688176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:35.688176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:35.688176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:35.688176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:35.688176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:35.688176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:35.688176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:35.688176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:35.688176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:35.688176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:35.688176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:35.688176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:35.689116 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:35.689116 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:35.689116 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:35.693080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 267/540] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:23:35.693080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b49421-dd59-4f17-bede-8ad210d56a19 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:35.790346 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:35.791609 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279252e-0809-4d8e-93a6-76264ab3f27a None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:35.807855 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b49421-dd59-4f17-bede-8ad210d56a19 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:35.827752 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff4690-525a-4ea2-bfe9-2330616c22da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:35.828379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff4690-525a-4ea2-bfe9-2330616c22da None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:35.828666 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff4690-525a-4ea2-bfe9-2330616c22da None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:35.828973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff4690-525a-4ea2-bfe9-2330616c22da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:35.839301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff4690-525a-4ea2-bfe9-2330616c22da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:35.843368 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229780ea-e947-4340-b977-5557989d2765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:35.843993 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229780ea-e947-4340-b977-5557989d2765 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:35.844269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229780ea-e947-4340-b977-5557989d2765 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:35.844537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229780ea-e947-4340-b977-5557989d2765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:35.844633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-24ff4690-525a-4ea2-bfe9-2330616c22da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:23:35.844633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:35.844633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:35.844633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:35.844633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:35.844633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:35.844633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:35.844633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:35.844633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:35.844633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:35.844633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:35.844633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:35.844633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:35.844633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:35.844633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:35.844633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:35.845714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:35.845714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:35.845714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:35.845714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:35.845714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:35.845714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:35.845714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:35.845714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:35.845714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:35.845714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:35.845714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:35.845714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:35.845714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:35.845714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:35.845714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:35.845714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:35.846529 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:35.846529 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:35.846529 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:35.846529 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:35.846529 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:35.846529 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:35.846529 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:35.846529 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:35.846529 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:35.846529 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:35.846529 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:23:35.846529 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:35.846529 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 274/541] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 20 13:23:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:35.846529 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff4690-525a-4ea2-bfe9-2330616c22da None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:35.859584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229780ea-e947-4340-b977-5557989d2765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:35.860978 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229780ea-e947-4340-b977-5557989d2765 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:35.874692 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:35.876886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-229780ea-e947-4340-b977-5557989d2765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:35.876886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:35.876886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:35.876886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:35.876886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:35.876886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:35.876886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:35.876886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:35.876886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:35.876886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:35.876886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:35.876886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:35.876886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:35.876886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:35.876886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:35.876886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:35.877877 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:35.877877 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:35.877877 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:35.877877 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:35.877877 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:35.877877 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:35.877877 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:35.877877 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:35.877877 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:35.877877 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:35.877877 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:35.877877 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:35.877877 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:35.877877 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:35.877877 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:35.877877 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:35.877877 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:35.878756 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:35.878756 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:35.878756 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:35.878756 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 268/542] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:35.879233 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33ecc9-6c9c-46b0-86cd-2f2554a9b087 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:35.879632 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33ecc9-6c9c-46b0-86cd-2f2554a9b087 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:35.879896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33ecc9-6c9c-46b0-86cd-2f2554a9b087 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:35.881161 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 269/543] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:35.881858 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33ecc9-6c9c-46b0-86cd-2f2554a9b087 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:35.954822 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff4690-525a-4ea2-bfe9-2330616c22da None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:35.983171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33ecc9-6c9c-46b0-86cd-2f2554a9b087 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:35.987854 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f6407-c48b-4732-90e8-733ed75e8ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:35.988612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f6407-c48b-4732-90e8-733ed75e8ffa None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:35.988916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f6407-c48b-4732-90e8-733ed75e8ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:35.989197 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f6407-c48b-4732-90e8-733ed75e8ffa None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:35.989830 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f6407-c48b-4732-90e8-733ed75e8ffa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:35.990128 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f6407-c48b-4732-90e8-733ed75e8ffa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:35.991510 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f6407-c48b-4732-90e8-733ed75e8ffa None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:36.004252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 275/544] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:36.005391 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6cfad9-0c40-4a0f-b747-cb2d35a41585 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:36.005685 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6cfad9-0c40-4a0f-b747-cb2d35a41585 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:36.005959 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6cfad9-0c40-4a0f-b747-cb2d35a41585 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:36.007162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 276/545] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:36.007856 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6cfad9-0c40-4a0f-b747-cb2d35a41585 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:36.020976 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81977d35-468a-4d0a-ae3e-e559033f5e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:36.021580 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81977d35-468a-4d0a-ae3e-e559033f5e1b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:36.021856 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81977d35-468a-4d0a-ae3e-e559033f5e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:36.022125 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81977d35-468a-4d0a-ae3e-e559033f5e1b None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:36.022787 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81977d35-468a-4d0a-ae3e-e559033f5e1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:36.023089 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81977d35-468a-4d0a-ae3e-e559033f5e1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:36.024441 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81977d35-468a-4d0a-ae3e-e559033f5e1b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:36.026584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:36.038780 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 270/546] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:23:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:36.038780 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c36989-7a37-4809-8078-665bd4276003 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:36.038780 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c36989-7a37-4809-8078-665bd4276003 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:36.038780 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c36989-7a37-4809-8078-665bd4276003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:36.053510 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-59c36989-7a37-4809-8078-665bd4276003 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:36.054705 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-59c36989-7a37-4809-8078-665bd4276003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:36.094876 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:36.116598 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6cfad9-0c40-4a0f-b747-cb2d35a41585 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:36.121733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 271/547] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:36.122565 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c36989-7a37-4809-8078-665bd4276003 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:36.166154 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d226a22c-216d-488a-94b7-e68b6b30d002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:36.166759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226a22c-216d-488a-94b7-e68b6b30d002 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:36.167079 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226a22c-216d-488a-94b7-e68b6b30d002 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:36.167291 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226a22c-216d-488a-94b7-e68b6b30d002 None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:36.167937 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d226a22c-216d-488a-94b7-e68b6b30d002 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:36.168235 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d226a22c-216d-488a-94b7-e68b6b30d002 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:36.169634 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d226a22c-216d-488a-94b7-e68b6b30d002 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:36.185255 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 277/548] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:36.185255 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab406b42-8d73-4b9f-9284-d148c289ee53 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:36.185255 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab406b42-8d73-4b9f-9284-d148c289ee53 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:36.185255 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab406b42-8d73-4b9f-9284-d148c289ee53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:36.200540 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-ab406b42-8d73-4b9f-9284-d148c289ee53 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:36.201751 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-ab406b42-8d73-4b9f-9284-d148c289ee53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:36.237489 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c36989-7a37-4809-8078-665bd4276003 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:36.266687 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:36.272628 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 278/549] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 20 13:23:36.273552 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab406b42-8d73-4b9f-9284-d148c289ee53 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:36.286428 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308d9b3-8477-4b61-b0a1-724e0f97efe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:36.287467 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308d9b3-8477-4b61-b0a1-724e0f97efe7 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:36.287748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308d9b3-8477-4b61-b0a1-724e0f97efe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:36.288019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308d9b3-8477-4b61-b0a1-724e0f97efe7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:36.296121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308d9b3-8477-4b61-b0a1-724e0f97efe7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:36.297611 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308d9b3-8477-4b61-b0a1-724e0f97efe7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:36.311300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d308d9b3-8477-4b61-b0a1-724e0f97efe7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 20 13:23:36.311300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:36.311300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:36.311300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:36.311300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:36.311300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:36.311300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:36.311300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:36.311300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:36.311300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:36.311300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:36.311300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:36.311300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:23:36.311300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:23:36.311300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:23:36.311300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:23:36.312222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:36.312222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:36.312222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:23:36.312222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:36.312222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:23:36.312222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:36.312222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:36.312222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:36.312222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:36.312222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:36.312222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:36.312222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:36.312222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:36.312222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:36.312222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:36.312222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:36.312222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:36.312222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:36.313207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:36.313207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:36.313207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:23:36.313207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:23:36.313207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:23:36.313207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:23:36.313207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:23:36.313207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:23:36.313207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 20 13:23:36.313207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:36.313207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 272/550] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Oct 20 13:23:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:36.313784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308d9b3-8477-4b61-b0a1-724e0f97efe7 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:36.376351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:36.412465 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab406b42-8d73-4b9f-9284-d148c289ee53 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:36.442085 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308d9b3-8477-4b61-b0a1-724e0f97efe7 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:36.447142 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa718b85-cb97-4af4-9967-d5d41ca4cda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:36.447768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa718b85-cb97-4af4-9967-d5d41ca4cda9 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:36.448043 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa718b85-cb97-4af4-9967-d5d41ca4cda9 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:36.448310 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa718b85-cb97-4af4-9967-d5d41ca4cda9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:36.458334 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa718b85-cb97-4af4-9967-d5d41ca4cda9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:36.459766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa718b85-cb97-4af4-9967-d5d41ca4cda9 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:36.474381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-fa718b85-cb97-4af4-9967-d5d41ca4cda9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:36.474381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:36.474381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:36.474381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:36.474381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:36.474381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:36.474381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:36.474381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:36.474381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:36.474381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:36.474381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:36.474381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:36.474381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:36.474381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:36.474381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:36.474381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:36.475319 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:36.475319 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:36.475319 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:36.475319 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:36.475319 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:36.475319 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:36.475319 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:36.475319 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:36.475319 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:36.475319 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:36.475319 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:36.475319 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:36.475319 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:36.475319 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:36.475319 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:36.475319 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:36.475319 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:36.476180 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:36.476180 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:36.476180 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:36.476180 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 279/551] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:36.476180 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa718b85-cb97-4af4-9967-d5d41ca4cda9 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:36.484701 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b0fee-a45f-4f4a-b4f1-1fbe66b69bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:36.485379 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b0fee-a45f-4f4a-b4f1-1fbe66b69bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:36.485656 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b0fee-a45f-4f4a-b4f1-1fbe66b69bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:36.485928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b0fee-a45f-4f4a-b4f1-1fbe66b69bc8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:36.496022 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b0fee-a45f-4f4a-b4f1-1fbe66b69bc8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:36.497525 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b0fee-a45f-4f4a-b4f1-1fbe66b69bc8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:36.514105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-9b6b0fee-a45f-4f4a-b4f1-1fbe66b69bc8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 20 13:23:36.514105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:36.514105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:36.514105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:36.514105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:36.514105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:36.514105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:36.514105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:36.514105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:36.514105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:36.514105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:36.514105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:36.514105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:23:36.514105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:23:36.514105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:23:36.514105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:23:36.515027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:36.515027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:36.515027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:23:36.515027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:36.515027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:23:36.515027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:36.515027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:36.515027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:36.515027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:36.515027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:36.515027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:36.515027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:36.515027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:36.515027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:36.515027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:36.515027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:36.515027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:36.515027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:36.515936 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:36.515936 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:36.515936 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:23:36.515936 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:23:36.515936 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:23:36.515936 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:23:36.515936 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:23:36.515936 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:23:36.515936 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 20 13:23:36.515936 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:36.515936 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 273/552] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 20 13:23:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:36.516506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b0fee-a45f-4f4a-b4f1-1fbe66b69bc8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:36.569268 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:36.592002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa718b85-cb97-4af4-9967-d5d41ca4cda9 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:36.614277 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:36.621474 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b0fee-a45f-4f4a-b4f1-1fbe66b69bc8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:36.625916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288cf704-aabb-4e51-a189-58b1266a48a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:36.626556 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cf704-aabb-4e51-a189-58b1266a48a4 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:36.626787 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cf704-aabb-4e51-a189-58b1266a48a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:36.627054 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cf704-aabb-4e51-a189-58b1266a48a4 None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:36.627770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288cf704-aabb-4e51-a189-58b1266a48a4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:36.628079 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288cf704-aabb-4e51-a189-58b1266a48a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:36.629653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288cf704-aabb-4e51-a189-58b1266a48a4 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:36.646540 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 280/553] 10.222.0.23 () {56 vars in 1149 bytes} [Thu Oct 20 13:23:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:36.647329 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288cf704-aabb-4e51-a189-58b1266a48a4 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:36.656051 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b560e94-8fbe-4d39-9f26-95fc67e1669c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:36.656690 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b560e94-8fbe-4d39-9f26-95fc67e1669c None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:36.657006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b560e94-8fbe-4d39-9f26-95fc67e1669c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:36.657281 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b560e94-8fbe-4d39-9f26-95fc67e1669c None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:36.657934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b560e94-8fbe-4d39-9f26-95fc67e1669c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:36.659284 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b560e94-8fbe-4d39-9f26-95fc67e1669c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:36.676850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 274/554] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 20 13:23:36 2022] POST /identity/v3/services => generated 251 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:23:36.677691 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b560e94-8fbe-4d39-9f26-95fc67e1669c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:36.762066 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288cf704-aabb-4e51-a189-58b1266a48a4 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:36.807511 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d4ba2-6a7f-4dac-8179-e830ad29f698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:36.808094 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b560e94-8fbe-4d39-9f26-95fc67e1669c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:36.808240 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d4ba2-6a7f-4dac-8179-e830ad29f698 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:36.808398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d4ba2-6a7f-4dac-8179-e830ad29f698 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:36.808658 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d4ba2-6a7f-4dac-8179-e830ad29f698 None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:36.809339 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d4ba2-6a7f-4dac-8179-e830ad29f698 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:36.809643 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d4ba2-6a7f-4dac-8179-e830ad29f698 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:36.810978 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:36.811099 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d4ba2-6a7f-4dac-8179-e830ad29f698 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:36.825802 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 281/555] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:36.826663 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d4ba2-6a7f-4dac-8179-e830ad29f698 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:36.855981 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759fafd-ea22-402c-b59f-a2ed259b5fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:36.856637 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759fafd-ea22-402c-b59f-a2ed259b5fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:36.856950 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759fafd-ea22-402c-b59f-a2ed259b5fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:36.857223 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759fafd-ea22-402c-b59f-a2ed259b5fa1 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:36.858034 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759fafd-ea22-402c-b59f-a2ed259b5fa1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:36.858333 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759fafd-ea22-402c-b59f-a2ed259b5fa1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:36.859763 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c759fafd-ea22-402c-b59f-a2ed259b5fa1 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:36.868177 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:36.883282 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 275/556] 10.222.0.23 () {56 vars in 1135 bytes} [Thu Oct 20 13:23:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:36.884067 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759fafd-ea22-402c-b59f-a2ed259b5fa1 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:36.939775 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d4ba2-6a7f-4dac-8179-e830ad29f698 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:36.973938 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b843bd-c36c-4edd-9b27-6deef169e121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:36.974681 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b843bd-c36c-4edd-9b27-6deef169e121 None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:36.974969 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b843bd-c36c-4edd-9b27-6deef169e121 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:36.975243 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b843bd-c36c-4edd-9b27-6deef169e121 None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:36.975902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b843bd-c36c-4edd-9b27-6deef169e121 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:36.977311 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b843bd-c36c-4edd-9b27-6deef169e121 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:36.993744 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c759fafd-ea22-402c-b59f-a2ed259b5fa1 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:36.996921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 282/557] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 20 13:23:36 2022] POST /identity/v3/services => generated 240 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:23:36.996921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b843bd-c36c-4edd-9b27-6deef169e121 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:37.051244 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8eb94b-f492-4769-8c26-e5204388decd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:37.052329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8eb94b-f492-4769-8c26-e5204388decd None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:37.052882 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8eb94b-f492-4769-8c26-e5204388decd None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:37.053300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8eb94b-f492-4769-8c26-e5204388decd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:37.065623 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8eb94b-f492-4769-8c26-e5204388decd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:37.068189 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8eb94b-f492-4769-8c26-e5204388decd None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:37.089329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-0f8eb94b-f492-4769-8c26-e5204388decd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:37.089329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:37.089329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:37.089329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:37.089329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:37.089329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:37.089329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:37.089329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:37.089329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:37.089329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:37.089329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:37.089329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:37.089329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:37.089329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:37.089329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:37.089329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:37.090499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:37.090499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:37.090499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:37.090499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:37.090499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:37.090499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:37.090499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:37.090499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:37.090499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:37.090499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:37.090499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:37.090499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:37.090499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:37.090499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:37.090499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:37.090499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:37.090499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:37.091620 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:37.091620 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:37.091620 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:37.094504 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 276/558] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:37.095720 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8eb94b-f492-4769-8c26-e5204388decd None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:37.112199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:37.149128 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:37.159819 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b843bd-c36c-4edd-9b27-6deef169e121 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:37.199243 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e9663-d168-460a-9c52-7a7ae2c52d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:37.200295 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e9663-d168-460a-9c52-7a7ae2c52d21 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:37.200768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e9663-d168-460a-9c52-7a7ae2c52d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:37.201288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e9663-d168-460a-9c52-7a7ae2c52d21 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:37.210016 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e9663-d168-460a-9c52-7a7ae2c52d21 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:37.211730 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e9663-d168-460a-9c52-7a7ae2c52d21 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:37.224968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-aa6e9663-d168-460a-9c52-7a7ae2c52d21 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 20 13:23:37.224968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:37.224968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:37.224968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:37.224968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:37.224968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:37.224968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:37.224968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:37.224968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:37.224968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:37.224968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:37.224968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:37.224968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:23:37.224968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:23:37.224968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:23:37.224968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:23:37.225917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:37.225917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:37.225917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:23:37.225917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:37.225917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:23:37.225917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:37.225917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:37.225917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:37.225917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:37.225917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:37.225917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:37.225917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:37.225917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:37.225917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:37.225917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:37.225917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:37.225917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:37.225917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:37.227417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:37.227417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:37.227417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:23:37.227417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:23:37.227417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:23:37.227417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:23:37.227417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:23:37.227417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:23:37.227417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 20 13:23:37.227417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:37.227417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 283/559] 10.222.0.23 () {56 vars in 1093 bytes} [Thu Oct 20 13:23:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:37.227417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e9663-d168-460a-9c52-7a7ae2c52d21 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:37.228053 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8eb94b-f492-4769-8c26-e5204388decd None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:37.291673 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606193d-56fa-4921-a12c-61cdeab7dbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:37.292584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606193d-56fa-4921-a12c-61cdeab7dbb5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:37.292971 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606193d-56fa-4921-a12c-61cdeab7dbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:37.293336 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606193d-56fa-4921-a12c-61cdeab7dbb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:37.321701 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0606193d-56fa-4921-a12c-61cdeab7dbb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:37.323437 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0606193d-56fa-4921-a12c-61cdeab7dbb5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:37.376562 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e9663-d168-460a-9c52-7a7ae2c52d21 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:37.378157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-0606193d-56fa-4921-a12c-61cdeab7dbb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:37.378157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:37.378157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:37.378157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:37.378157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:37.378157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:37.378157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:37.378157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:37.378157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:37.378157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:37.378157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:37.378157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:37.378157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:37.378157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:37.378157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:37.378157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:37.379265 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:37.379265 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:37.379265 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:37.379265 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:37.379265 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:37.379265 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:37.379265 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:37.379265 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:37.379265 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:37.379265 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:37.379265 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:37.379265 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:37.379265 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:37.379265 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:37.379265 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:37.379265 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:37.379265 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:37.380165 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:37.380165 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:37.380165 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:37.380165 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 277/560] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:37.380400 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606193d-56fa-4921-a12c-61cdeab7dbb5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:37.439666 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa9016-6db5-42b8-b8a0-c340099cd956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:37.440619 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa9016-6db5-42b8-b8a0-c340099cd956 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:37.441093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa9016-6db5-42b8-b8a0-c340099cd956 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:37.442034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa9016-6db5-42b8-b8a0-c340099cd956 None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:37.443465 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa9016-6db5-42b8-b8a0-c340099cd956 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:37.444009 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa9016-6db5-42b8-b8a0-c340099cd956 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:37.445882 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa9016-6db5-42b8-b8a0-c340099cd956 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:37.463661 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 284/561] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:37.464919 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa9016-6db5-42b8-b8a0-c340099cd956 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:37.487297 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:37.574420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:37.611330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa9016-6db5-42b8-b8a0-c340099cd956 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:37.623891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606193d-56fa-4921-a12c-61cdeab7dbb5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:37.665373 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066908e3-6911-4061-ae0d-b92135a3c813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:37.666958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066908e3-6911-4061-ae0d-b92135a3c813 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:37.667466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066908e3-6911-4061-ae0d-b92135a3c813 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:37.667943 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066908e3-6911-4061-ae0d-b92135a3c813 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:37.668932 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066908e3-6911-4061-ae0d-b92135a3c813 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:37.669443 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066908e3-6911-4061-ae0d-b92135a3c813 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:37.671700 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066908e3-6911-4061-ae0d-b92135a3c813 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:37.693073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 278/562] 10.222.0.23 () {56 vars in 1133 bytes} [Thu Oct 20 13:23:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:37.694260 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066908e3-6911-4061-ae0d-b92135a3c813 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:37.709500 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f48e29c-3905-405b-8aec-1f31a857082c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:37.710191 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48e29c-3905-405b-8aec-1f31a857082c None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:37.710386 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48e29c-3905-405b-8aec-1f31a857082c None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:37.710658 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48e29c-3905-405b-8aec-1f31a857082c None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:37.711489 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f48e29c-3905-405b-8aec-1f31a857082c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:37.711610 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f48e29c-3905-405b-8aec-1f31a857082c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:37.718556 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f48e29c-3905-405b-8aec-1f31a857082c None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:37.746726 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 285/563] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:37.747802 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f48e29c-3905-405b-8aec-1f31a857082c None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:37.819630 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066908e3-6911-4061-ae0d-b92135a3c813 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:37.844245 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:37.866363 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:37.868398 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa9dae-73d1-48f4-b911-c774eead0f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:37.869604 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa9dae-73d1-48f4-b911-c774eead0f17 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:37.870084 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa9dae-73d1-48f4-b911-c774eead0f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:37.870521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa9dae-73d1-48f4-b911-c774eead0f17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:37.880403 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa9dae-73d1-48f4-b911-c774eead0f17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:37.882448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa9dae-73d1-48f4-b911-c774eead0f17 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:37.890854 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f48e29c-3905-405b-8aec-1f31a857082c None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:37.898360 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-4afa9dae-73d1-48f4-b911-c774eead0f17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:37.898360 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:37.898360 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:37.898360 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:37.898360 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:37.898360 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:37.898360 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:37.898360 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:37.898360 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:37.898360 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:37.898360 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:37.898360 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:37.898360 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:23:37.898360 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:23:37.898360 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:23:37.899277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:23:37.899277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:37.899277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:37.899277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:37.899277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:37.899277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:37.899277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:37.899277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:37.899277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:37.899277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:37.899277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:37.899277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:37.899277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:37.899277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:37.899277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:37.899277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:37.899277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:37.900150 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:23:37.900150 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:23:37.900150 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:23:37.900150 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:23:37.900150 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:23:37.900150 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:23:37.900150 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:37.900150 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:37.907129 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 279/564] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:37.907129 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa9dae-73d1-48f4-b911-c774eead0f17 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:37.934237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fa652-8268-496a-a57a-27bf8e9588cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:37.934842 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fa652-8268-496a-a57a-27bf8e9588cf None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:37.935117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fa652-8268-496a-a57a-27bf8e9588cf None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:37.935386 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fa652-8268-496a-a57a-27bf8e9588cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:38.006910 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3fa652-8268-496a-a57a-27bf8e9588cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:38.008268 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3fa652-8268-496a-a57a-27bf8e9588cf None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:38.039404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa9dae-73d1-48f4-b911-c774eead0f17 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:38.083566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd31a0a-0233-4300-94c1-8068ca39f22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:38.084200 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd31a0a-0233-4300-94c1-8068ca39f22d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:38.084476 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd31a0a-0233-4300-94c1-8068ca39f22d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:38.085607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd31a0a-0233-4300-94c1-8068ca39f22d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:38.093227 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:38.094345 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd31a0a-0233-4300-94c1-8068ca39f22d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:38.096514 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd31a0a-0233-4300-94c1-8068ca39f22d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:38.112526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-9cd31a0a-0233-4300-94c1-8068ca39f22d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:38.112526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:38.112526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:38.112526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:38.112526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:38.112526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:38.112526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:38.112526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:38.112526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:38.112526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:38.112526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:38.112526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:38.112526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:23:38.112526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:23:38.112526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:23:38.114348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:23:38.114348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:38.114348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:38.114348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:38.114348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:38.114348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:38.114348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:38.114348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:38.114348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:38.114348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:38.114348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:38.114348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:38.114348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:38.114348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:38.114348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:38.114348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:38.114348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:38.115329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:23:38.115329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:23:38.115329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:23:38.115329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:23:38.115329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:23:38.115329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:23:38.115329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:38.115329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:38.117376 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 280/565] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:38.118355 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd31a0a-0233-4300-94c1-8068ca39f22d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:38.136200 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 286/566] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:23:37 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 13:23:38.137309 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fa652-8268-496a-a57a-27bf8e9588cf None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:38.171440 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:38.246537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd31a0a-0233-4300-94c1-8068ca39f22d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:38.255484 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fa652-8268-496a-a57a-27bf8e9588cf None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:38.282775 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ef017-0908-4cb0-bbde-426dca251040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:38.283321 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ef017-0908-4cb0-bbde-426dca251040 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:38.283606 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ef017-0908-4cb0-bbde-426dca251040 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:38.283880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ef017-0908-4cb0-bbde-426dca251040 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:38.284572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ef017-0908-4cb0-bbde-426dca251040 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:38.285888 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ef017-0908-4cb0-bbde-426dca251040 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:38.286991 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ef017-0908-4cb0-bbde-426dca251040 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:38.307063 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 281/567] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 20 13:23:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:38.307063 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ef017-0908-4cb0-bbde-426dca251040 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:38.313400 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a13f1ea-2657-4766-aba1-9f2757f8ee96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:38.314292 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a13f1ea-2657-4766-aba1-9f2757f8ee96 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:38.314787 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a13f1ea-2657-4766-aba1-9f2757f8ee96 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:38.315264 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a13f1ea-2657-4766-aba1-9f2757f8ee96 None admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:38.316162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a13f1ea-2657-4766-aba1-9f2757f8ee96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:38.316725 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a13f1ea-2657-4766-aba1-9f2757f8ee96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:38.318482 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a13f1ea-2657-4766-aba1-9f2757f8ee96 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:38.331315 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 287/568] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 20 13:23:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:38.332454 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a13f1ea-2657-4766-aba1-9f2757f8ee96 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:38.374628 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:38.434858 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ef017-0908-4cb0-bbde-426dca251040 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:38.446097 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a13f1ea-2657-4766-aba1-9f2757f8ee96 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:38.450306 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:38.482539 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de201ef-e24b-436a-aa4e-cbb3b1ebf324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:38.483271 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de201ef-e24b-436a-aa4e-cbb3b1ebf324 None admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:38.483615 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de201ef-e24b-436a-aa4e-cbb3b1ebf324 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:38.484001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de201ef-e24b-436a-aa4e-cbb3b1ebf324 None admin] PATH_INFO: `/v3/projects/4a81b6ecfb6e46a8b08c8bc2b4000fc7/users/85bb137d0fb74031a49a5c629b32c32b/roles/0f01a310371a4f6bbae7bb2e319a3b39` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:38.488626 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f5357-8a0f-4c52-9e32-e4100ca95d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:38.489641 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f5357-8a0f-4c52-9e32-e4100ca95d7e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:38.490143 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f5357-8a0f-4c52-9e32-e4100ca95d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:38.490618 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f5357-8a0f-4c52-9e32-e4100ca95d7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:38.499698 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de201ef-e24b-436a-aa4e-cbb3b1ebf324 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a81b6ecfb6e46a8b08c8bc2b4000fc7, user_id=85bb137d0fb74031a49a5c629b32c32b, role_id=0f01a310371a4f6bbae7bb2e319a3b39)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:38.501293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de201ef-e24b-436a-aa4e-cbb3b1ebf324 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:38.505633 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f5357-8a0f-4c52-9e32-e4100ca95d7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:38.507660 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f5357-8a0f-4c52-9e32-e4100ca95d7e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:38.523706 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 288/569] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:23:38 2022] PUT /identity/v3/projects/4a81b6ecfb6e46a8b08c8bc2b4000fc7/users/85bb137d0fb74031a49a5c629b32c32b/roles/0f01a310371a4f6bbae7bb2e319a3b39 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:23:38.524953 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de201ef-e24b-436a-aa4e-cbb3b1ebf324 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:38.529908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-f15f5357-8a0f-4c52-9e32-e4100ca95d7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:38.529908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:38.529908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:38.529908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:38.529908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:38.529908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:38.529908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:38.529908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:38.529908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:38.529908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:38.529908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:38.529908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:38.529908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:38.529908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:38.529908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:38.529908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:38.531066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:38.531066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:38.531066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:38.531066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:38.531066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:38.531066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:38.531066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:38.531066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:38.531066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:38.531066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:38.531066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:38.531066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:38.531066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:38.531066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:38.531066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:38.531066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:38.531066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:38.531969 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:38.531969 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:38.531969 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:38.536923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 282/570] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:38.538723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f5357-8a0f-4c52-9e32-e4100ca95d7e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:38.677359 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f5357-8a0f-4c52-9e32-e4100ca95d7e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:38.680020 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:38.710009 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de201ef-e24b-436a-aa4e-cbb3b1ebf324 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:38.730736 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415a9e3-10d4-4527-a21a-bf5609ee1414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:38.731721 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415a9e3-10d4-4527-a21a-bf5609ee1414 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:38.732159 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415a9e3-10d4-4527-a21a-bf5609ee1414 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:38.732577 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415a9e3-10d4-4527-a21a-bf5609ee1414 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:38.733413 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415a9e3-10d4-4527-a21a-bf5609ee1414 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:38.733863 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415a9e3-10d4-4527-a21a-bf5609ee1414 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:38.735506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415a9e3-10d4-4527-a21a-bf5609ee1414 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:38.749677 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 283/571] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:38.751274 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aee0db-182d-408e-80e4-ae1935260ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:38.752034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aee0db-182d-408e-80e4-ae1935260ddb None admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:38.752408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aee0db-182d-408e-80e4-ae1935260ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:38.752780 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aee0db-182d-408e-80e4-ae1935260ddb None admin] PATH_INFO: `/v3/projects/4a81b6ecfb6e46a8b08c8bc2b4000fc7/users/9834211488354651803000c7e8fab80b/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:38.755458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aee0db-182d-408e-80e4-ae1935260ddb None admin] RBAC: Authorizing `identity:create_grant(project_id=4a81b6ecfb6e46a8b08c8bc2b4000fc7, user_id=9834211488354651803000c7e8fab80b, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:38.755743 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415a9e3-10d4-4527-a21a-bf5609ee1414 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:38.757171 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aee0db-182d-408e-80e4-ae1935260ddb None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:38.773558 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:38.778669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 289/572] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:23:38 2022] PUT /identity/v3/projects/4a81b6ecfb6e46a8b08c8bc2b4000fc7/users/9834211488354651803000c7e8fab80b/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:23:38.889184 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415a9e3-10d4-4527-a21a-bf5609ee1414 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:38.944688 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f709f-eb1e-484f-b222-827255f4b1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:38.945790 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f709f-eb1e-484f-b222-827255f4b1a8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:38.946299 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f709f-eb1e-484f-b222-827255f4b1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:38.946816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f709f-eb1e-484f-b222-827255f4b1a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:38.954373 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f709f-eb1e-484f-b222-827255f4b1a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:38.956127 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f709f-eb1e-484f-b222-827255f4b1a8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:38.971835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-644f709f-eb1e-484f-b222-827255f4b1a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:38.971835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:38.971835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:38.971835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:38.971835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:38.971835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:38.971835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:38.971835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:38.971835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:38.971835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:38.971835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:38.971835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:38.971835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:23:38.971835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:23:38.971835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:23:38.972958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:23:38.972958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:38.972958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:38.972958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:38.972958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:38.972958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:38.972958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:38.972958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:38.972958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:38.972958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:38.972958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:38.972958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:38.972958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:38.972958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:38.972958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:38.972958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:38.972958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:38.973977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:23:38.973977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:23:38.973977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:23:38.973977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:23:38.973977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:23:38.973977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:23:38.973977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:38.973977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:38.976655 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 284/573] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:38.982210 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aee0db-182d-408e-80e4-ae1935260ddb None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:39.136553 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aee0db-182d-408e-80e4-ae1935260ddb None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:39.178206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573c42f-27c9-4927-b135-f2a47512e01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:39.178781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573c42f-27c9-4927-b135-f2a47512e01a None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:39.179063 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573c42f-27c9-4927-b135-f2a47512e01a None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:39.179332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573c42f-27c9-4927-b135-f2a47512e01a None admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:39.180089 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573c42f-27c9-4927-b135-f2a47512e01a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:39.180396 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573c42f-27c9-4927-b135-f2a47512e01a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:39.185456 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573c42f-27c9-4927-b135-f2a47512e01a None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:39.197702 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 290/574] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 20 13:23:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:39.203461 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f709f-eb1e-484f-b222-827255f4b1a8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:39.320513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:39.343850 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d7639-955d-4269-865b-1cb9ddce4108 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:39.344145 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d7639-955d-4269-865b-1cb9ddce4108 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:39.344395 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d7639-955d-4269-865b-1cb9ddce4108 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:39.346040 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 291/575] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:39.347059 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed067ba9-73fb-4015-8dea-e6dcc548ccf9 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:39.347336 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed067ba9-73fb-4015-8dea-e6dcc548ccf9 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:39.347648 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed067ba9-73fb-4015-8dea-e6dcc548ccf9 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:39.349105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 292/576] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:39.354209 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4deafda-b236-4bf8-aaac-4864ccca086c None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:39.354566 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4deafda-b236-4bf8-aaac-4864ccca086c None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:39.354838 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4deafda-b236-4bf8-aaac-4864ccca086c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:39.360175 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f709f-eb1e-484f-b222-827255f4b1a8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:39.369452 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:39.375556 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-d4deafda-b236-4bf8-aaac-4864ccca086c None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:39.377306 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-d4deafda-b236-4bf8-aaac-4864ccca086c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:39.404576 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d9b0b-a9a4-4e30-ab2e-0e6ebe37ed24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:39.405216 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d9b0b-a9a4-4e30-ab2e-0e6ebe37ed24 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:39.405572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d9b0b-a9a4-4e30-ab2e-0e6ebe37ed24 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:39.406102 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d9b0b-a9a4-4e30-ab2e-0e6ebe37ed24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:39.407255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d9b0b-a9a4-4e30-ab2e-0e6ebe37ed24 None admin] RBAC: Adding query filter params (role.id=2bcf35e1bc5f4634b0aa6411fad2f43f, scope.project.id=4a81b6ecfb6e46a8b08c8bc2b4000fc7, user.id=ee5c2a3855cb4a668177aa19f337778d) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:39.407850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d9b0b-a9a4-4e30-ab2e-0e6ebe37ed24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:39.409625 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d9b0b-a9a4-4e30-ab2e-0e6ebe37ed24 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:39.417181 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 285/577] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:23:39 2022] GET /identity/v3/role_assignments?user.id=ee5c2a3855cb4a668177aa19f337778d&scope.project.id=4a81b6ecfb6e46a8b08c8bc2b4000fc7&role.id=2bcf35e1bc5f4634b0aa6411fad2f43f => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:39.418445 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449260a-192f-49c6-924f-c4ff1dd07fa8 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:39.418730 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449260a-192f-49c6-924f-c4ff1dd07fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:39.419008 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449260a-192f-49c6-924f-c4ff1dd07fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:39.448548 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c449260a-192f-49c6-924f-c4ff1dd07fa8 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:39.449953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c449260a-192f-49c6-924f-c4ff1dd07fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:39.518586 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 293/578] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 13:23:39.553256 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 286/579] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:23:39.830566 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4deafda-b236-4bf8-aaac-4864ccca086c None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:39.876596 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449260a-192f-49c6-924f-c4ff1dd07fa8 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:39.952800 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:39.994551 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449260a-192f-49c6-924f-c4ff1dd07fa8 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:39.999623 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4deafda-b236-4bf8-aaac-4864ccca086c None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:40.033772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b07fbc-45dc-46ac-8a6b-da1caff3ec8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:40.034416 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b07fbc-45dc-46ac-8a6b-da1caff3ec8d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:40.034697 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b07fbc-45dc-46ac-8a6b-da1caff3ec8d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:40.035005 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b07fbc-45dc-46ac-8a6b-da1caff3ec8d None admin] PATH_INFO: `/v3/services/network` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:40.035646 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b07fbc-45dc-46ac-8a6b-da1caff3ec8d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:40.037050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b07fbc-45dc-46ac-8a6b-da1caff3ec8d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:40.049142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-f0b07fbc-45dc-46ac-8a6b-da1caff3ec8d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 13:23:40.049142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:40.049142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:23:40.049142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:23:40.049142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:23:40.049142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:23:40.049142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:23:40.049142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:40.049142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 13:23:40.049142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:40.049142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:23:40.049142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:40.049142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:40.049142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:40.049142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:40.049142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:40.049142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:40.050199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:40.050199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:40.050199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:40.050199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:40.050199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:40.050199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:40.050199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:23:40.050199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:23:40.050199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:23:40.050199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:23:40.050199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:40.050199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:40.050199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:40.050199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:40.050199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:40.050199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:40.051305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:40.051305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:40.051305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:40.051305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:40.051305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:40.051305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:40.051305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:40.051305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:40.051305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:40.051305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:40.051305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:23:40.051305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:40.051305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 13:23:40.051305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:40.051305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 287/580] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:39 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:40.057227 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b07fbc-45dc-46ac-8a6b-da1caff3ec8d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:40.079644 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96d7a9-dfc5-41db-bd30-035f12251e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:40.082452 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96d7a9-dfc5-41db-bd30-035f12251e14 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:40.083642 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96d7a9-dfc5-41db-bd30-035f12251e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:40.083922 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96d7a9-dfc5-41db-bd30-035f12251e14 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:40.084521 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96d7a9-dfc5-41db-bd30-035f12251e14 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:40.089076 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96d7a9-dfc5-41db-bd30-035f12251e14 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:40.103117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-5d96d7a9-dfc5-41db-bd30-035f12251e14 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 13:23:40.103117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:40.103117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:23:40.103117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:23:40.103117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:23:40.103117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:23:40.103117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:23:40.103117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:40.103117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 13:23:40.103117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:40.103117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:23:40.103117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:40.103117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:40.103117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:40.103117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:40.103117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:40.103117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:40.104160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:40.104160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:40.104160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:40.104160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:40.104160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:40.104160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:40.104160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:23:40.104160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:23:40.104160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:23:40.104160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:23:40.104160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:40.104160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:40.104160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:40.104160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:40.104160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:40.104160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:40.105078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:40.105078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:40.105078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:40.105078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:40.105078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:40.105078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:40.105078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:40.105078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:40.105078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:40.105078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:40.105078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:23:40.105078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:40.105078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 13:23:40.105078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:40.105078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 294/581] 10.222.0.23 () {56 vars in 1129 bytes} [Thu Oct 20 13:23:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:40.116897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:40.116897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96d7a9-dfc5-41db-bd30-035f12251e14 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:40.207150 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b07fbc-45dc-46ac-8a6b-da1caff3ec8d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:40.239565 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:40.247554 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00915f4-2b7d-48f0-af9f-52fc36325d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:40.248177 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00915f4-2b7d-48f0-af9f-52fc36325d4d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:40.248461 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00915f4-2b7d-48f0-af9f-52fc36325d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:40.248740 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00915f4-2b7d-48f0-af9f-52fc36325d4d None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:40.249407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00915f4-2b7d-48f0-af9f-52fc36325d4d None admin] RBAC: Adding query filter params (name=network) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:40.249714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00915f4-2b7d-48f0-af9f-52fc36325d4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:40.251114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00915f4-2b7d-48f0-af9f-52fc36325d4d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:40.260457 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 288/582] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 20 13:23:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:40.267407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00915f4-2b7d-48f0-af9f-52fc36325d4d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:40.283264 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96d7a9-dfc5-41db-bd30-035f12251e14 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:40.352943 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbad4db-f06f-4a35-8323-3a5e5a1dc032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:40.353489 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbad4db-f06f-4a35-8323-3a5e5a1dc032 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:40.353812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbad4db-f06f-4a35-8323-3a5e5a1dc032 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:40.354086 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbad4db-f06f-4a35-8323-3a5e5a1dc032 None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:40.354719 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbad4db-f06f-4a35-8323-3a5e5a1dc032 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:40.355019 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbad4db-f06f-4a35-8323-3a5e5a1dc032 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:40.356381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbad4db-f06f-4a35-8323-3a5e5a1dc032 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:40.366950 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 295/583] 10.222.0.23 () {56 vars in 1124 bytes} [Thu Oct 20 13:23:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:40.372425 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbad4db-f06f-4a35-8323-3a5e5a1dc032 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:40.399096 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00915f4-2b7d-48f0-af9f-52fc36325d4d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:40.434455 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc72074-3620-4cf1-ba29-6f6ca4deab15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:40.435069 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc72074-3620-4cf1-ba29-6f6ca4deab15 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:40.435350 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc72074-3620-4cf1-ba29-6f6ca4deab15 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:40.435623 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc72074-3620-4cf1-ba29-6f6ca4deab15 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:40.436260 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc72074-3620-4cf1-ba29-6f6ca4deab15 None admin] RBAC: Adding query filter params (type=network) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:40.436563 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc72074-3620-4cf1-ba29-6f6ca4deab15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:40.438000 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc72074-3620-4cf1-ba29-6f6ca4deab15 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:40.443609 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 289/584] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 20 13:23:40 2022] GET /identity/v3/services?type=network => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:23:40.449822 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc72074-3620-4cf1-ba29-6f6ca4deab15 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:40.509859 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:40.513660 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:40.557892 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc72074-3620-4cf1-ba29-6f6ca4deab15 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:40.593102 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b2c18-c4ef-45b8-b9e0-4e7341267c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:40.593716 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b2c18-c4ef-45b8-b9e0-4e7341267c5f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:40.594000 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b2c18-c4ef-45b8-b9e0-4e7341267c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:40.594274 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b2c18-c4ef-45b8-b9e0-4e7341267c5f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:40.594974 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652b2c18-c4ef-45b8-b9e0-4e7341267c5f None admin] RBAC: Adding query filter params (interface=public, service_id=098c57bd0e7140c68c2f3666d576021e, region_id=RegionOne) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:40.595283 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652b2c18-c4ef-45b8-b9e0-4e7341267c5f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:40.596649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652b2c18-c4ef-45b8-b9e0-4e7341267c5f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:40.605358 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 290/585] 10.222.0.23 () {56 vars in 1251 bytes} [Thu Oct 20 13:23:40 2022] GET /identity/v3/endpoints?service_id=098c57bd0e7140c68c2f3666d576021e&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:23:40.610089 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbad4db-f06f-4a35-8323-3a5e5a1dc032 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:40.613101 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b2c18-c4ef-45b8-b9e0-4e7341267c5f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:40.676264 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8302e-cb3a-4849-ad39-094981a47edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:40.676934 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8302e-cb3a-4849-ad39-094981a47edc None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:40.677217 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8302e-cb3a-4849-ad39-094981a47edc None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:40.677488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8302e-cb3a-4849-ad39-094981a47edc None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:40.678119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8302e-cb3a-4849-ad39-094981a47edc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:40.678419 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8302e-cb3a-4849-ad39-094981a47edc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:40.679995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8302e-cb3a-4849-ad39-094981a47edc None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:40.694774 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 296/586] 10.222.0.23 () {56 vars in 1124 bytes} [Thu Oct 20 13:23:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:23:40.701829 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8302e-cb3a-4849-ad39-094981a47edc None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:40.764144 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b2c18-c4ef-45b8-b9e0-4e7341267c5f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:40.793253 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:40.801882 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacbddb-0718-4495-93d4-e157dc4e2404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:40.802599 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacbddb-0718-4495-93d4-e157dc4e2404 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:40.802883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacbddb-0718-4495-93d4-e157dc4e2404 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:40.803241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacbddb-0718-4495-93d4-e157dc4e2404 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:40.803861 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aacbddb-0718-4495-93d4-e157dc4e2404 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:40.807398 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aacbddb-0718-4495-93d4-e157dc4e2404 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:40.825032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 291/587] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Oct 20 13:23:40 2022] GET /identity/v3/services => generated 772 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:40.912024 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8302e-cb3a-4849-ad39-094981a47edc None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:40.949460 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:40.962249 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7c619-ef50-4a62-a95f-5cdfc2f988f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:40.962875 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7c619-ef50-4a62-a95f-5cdfc2f988f7 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:40.963158 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7c619-ef50-4a62-a95f-5cdfc2f988f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:40.963435 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7c619-ef50-4a62-a95f-5cdfc2f988f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:40.964147 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe7c619-ef50-4a62-a95f-5cdfc2f988f7 None admin] RBAC: Adding query filter params (interface=public, service_id=55dfe90c1bf04828aa016a020e9818b2, region_id=RegionOne) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:40.964457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe7c619-ef50-4a62-a95f-5cdfc2f988f7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:40.965875 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe7c619-ef50-4a62-a95f-5cdfc2f988f7 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:40.979372 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 297/588] 10.222.0.23 () {56 vars in 1251 bytes} [Thu Oct 20 13:23:40 2022] GET /identity/v3/endpoints?service_id=55dfe90c1bf04828aa016a020e9818b2&interface=public®ion_id=RegionOne => generated 194 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:40.985515 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacbddb-0718-4495-93d4-e157dc4e2404 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:41.156729 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacbddb-0718-4495-93d4-e157dc4e2404 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:41.197111 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e283374-4de2-4b56-94b1-7c8eb970570a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:41.197631 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e283374-4de2-4b56-94b1-7c8eb970570a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:41.197914 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e283374-4de2-4b56-94b1-7c8eb970570a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:41.198190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e283374-4de2-4b56-94b1-7c8eb970570a None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:41.198808 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e283374-4de2-4b56-94b1-7c8eb970570a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:41.200543 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e283374-4de2-4b56-94b1-7c8eb970570a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:41.207092 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 292/589] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Oct 20 13:23:40 2022] GET /identity/v3/services => generated 772 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:41.229756 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7105a1-85b0-4923-9e4f-de13e6094db7 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:41.230040 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7105a1-85b0-4923-9e4f-de13e6094db7 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:41.230435 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7105a1-85b0-4923-9e4f-de13e6094db7 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:41.232110 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 298/590] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:41.239918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfd4b4-bcdd-4740-a977-f3630ee7f4eb None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:41.240306 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfd4b4-bcdd-4740-a977-f3630ee7f4eb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:41.240584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfd4b4-bcdd-4740-a977-f3630ee7f4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:41.264130 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-dedfd4b4-bcdd-4740-a977-f3630ee7f4eb None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:41.265480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-dedfd4b4-bcdd-4740-a977-f3630ee7f4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:41.333177 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:41.365830 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 293/591] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 13:23:41.643404 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f161142-2f7b-4b28-9194-c654f8a00e23 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:41.644042 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f161142-2f7b-4b28-9194-c654f8a00e23 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:41.645103 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f161142-2f7b-4b28-9194-c654f8a00e23 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:41.647020 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 299/592] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:41.652810 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26070c2-1e0e-4d65-9226-6b1e7bae46ce None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:41.653403 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26070c2-1e0e-4d65-9226-6b1e7bae46ce None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:41.653888 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26070c2-1e0e-4d65-9226-6b1e7bae46ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:41.683556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f26070c2-1e0e-4d65-9226-6b1e7bae46ce None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:41.685230 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f26070c2-1e0e-4d65-9226-6b1e7bae46ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:41.724363 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f161142-2f7b-4b28-9194-c654f8a00e23 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:41.784798 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 294/593] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 13:23:41.910349 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f161142-2f7b-4b28-9194-c654f8a00e23 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:41.953571 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921741c-86d1-4431-8749-c1274cefd5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:41.954221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921741c-86d1-4431-8749-c1274cefd5d6 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:41.954463 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921741c-86d1-4431-8749-c1274cefd5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:41.954729 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921741c-86d1-4431-8749-c1274cefd5d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:41.973836 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921741c-86d1-4431-8749-c1274cefd5d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:41.979630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-d921741c-86d1-4431-8749-c1274cefd5d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:23:41.979630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:41.979630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:41.979630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:41.979630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:41.979630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:41.979630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:41.979630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:41.979630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:41.979630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:41.979630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:41.979630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:41.979630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:41.979630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:41.979630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:41.979630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:41.980562 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:41.980562 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:41.980562 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:41.980562 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:41.980562 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:41.980562 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:41.980562 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:41.980562 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:41.980562 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:41.980562 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:41.980562 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:41.980562 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:41.980562 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:41.980562 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:41.980562 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:41.980562 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:41.981420 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:41.981420 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:41.981420 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:41.981420 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:41.981420 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:41.981420 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:41.981420 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:41.981420 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:41.981420 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:41.981420 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:41.981420 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:23:41.981420 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:41.983768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 300/594] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 20 13:23:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:41.990008 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26070c2-1e0e-4d65-9226-6b1e7bae46ce None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:42.102103 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921741c-86d1-4431-8749-c1274cefd5d6 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:42.163230 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26070c2-1e0e-4d65-9226-6b1e7bae46ce None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:42.183661 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:42.196405 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:42.212510 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78420bdc-5373-459c-b61e-b4ba723d35f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:42.213407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78420bdc-5373-459c-b61e-b4ba723d35f9 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:42.213851 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78420bdc-5373-459c-b61e-b4ba723d35f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:42.214282 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78420bdc-5373-459c-b61e-b4ba723d35f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:42.215089 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78420bdc-5373-459c-b61e-b4ba723d35f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:42.215551 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78420bdc-5373-459c-b61e-b4ba723d35f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:42.217093 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78420bdc-5373-459c-b61e-b4ba723d35f9 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:42.228336 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 295/595] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 20 13:23:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:42.229937 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f56ab8-63fb-4de1-a89e-11226a1bde5e None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:42.230325 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f56ab8-63fb-4de1-a89e-11226a1bde5e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:42.230703 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f56ab8-63fb-4de1-a89e-11226a1bde5e None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:42.232133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 296/596] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:42.237510 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f56ab8-63fb-4de1-a89e-11226a1bde5e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:42.258784 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921741c-86d1-4431-8749-c1274cefd5d6 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:42.305912 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d483d7-b3b0-4ba8-b7f9-5b552fdc60e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:42.306914 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d483d7-b3b0-4ba8-b7f9-5b552fdc60e3 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:42.307355 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d483d7-b3b0-4ba8-b7f9-5b552fdc60e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:42.307881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d483d7-b3b0-4ba8-b7f9-5b552fdc60e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:42.319736 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d483d7-b3b0-4ba8-b7f9-5b552fdc60e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:42.325653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-c2d483d7-b3b0-4ba8-b7f9-5b552fdc60e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:23:42.325653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:42.325653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:42.325653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:42.325653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:42.325653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:42.325653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:42.325653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:42.325653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:42.325653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:42.325653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:42.325653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:42.325653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:42.325653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:42.325653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:42.325653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:42.326693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:42.326693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:42.326693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:42.326693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:42.326693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:42.326693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:42.326693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:42.326693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:42.326693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:42.326693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:42.326693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:42.326693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:42.326693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:42.326693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:42.326693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:42.326693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:42.327534 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:42.327534 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:42.327534 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:42.327534 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:42.327534 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:42.327534 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:42.327534 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:42.327534 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:42.327534 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:42.327534 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:42.327534 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:23:42.327534 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:42.330275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 301/597] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 20 13:23:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:23:42.331630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c4f63-6886-4d0c-a303-861d761e8d9b None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:42.333193 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c4f63-6886-4d0c-a303-861d761e8d9b None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:42.333637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c4f63-6886-4d0c-a303-861d761e8d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:42.353564 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-cf1c4f63-6886-4d0c-a303-861d761e8d9b None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:42.355146 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-cf1c4f63-6886-4d0c-a303-861d761e8d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:42.388296 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f56ab8-63fb-4de1-a89e-11226a1bde5e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:42.435444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 302/598] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:23:42.436677 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c4f63-6886-4d0c-a303-861d761e8d9b None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:42.436786 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7503adf-1707-422a-978d-f5ecbf5d7731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:42.437397 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7503adf-1707-422a-978d-f5ecbf5d7731 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:42.437856 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7503adf-1707-422a-978d-f5ecbf5d7731 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:42.438270 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7503adf-1707-422a-978d-f5ecbf5d7731 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:42.449740 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7503adf-1707-422a-978d-f5ecbf5d7731 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:42.451440 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7503adf-1707-422a-978d-f5ecbf5d7731 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:42.467080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-f7503adf-1707-422a-978d-f5ecbf5d7731 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:42.467080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:42.467080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:42.467080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:42.467080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:42.467080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:42.467080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:42.467080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:42.467080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:42.467080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:42.467080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:42.467080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:42.467080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:42.467080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:42.467080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:42.467080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:42.468042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:42.468042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:42.468042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:42.468042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:42.468042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:42.468042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:42.468042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:42.468042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:42.468042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:42.468042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:42.468042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:42.468042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:42.468042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:42.468042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:42.468042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:42.468042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:42.468042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:42.468978 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:42.468978 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:42.468978 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:42.475312 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 297/599] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:23:42.476628 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7503adf-1707-422a-978d-f5ecbf5d7731 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:42.503563 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:42.549906 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:42.639558 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7503adf-1707-422a-978d-f5ecbf5d7731 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:42.656459 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c4f63-6886-4d0c-a303-861d761e8d9b None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:42.714605 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28158eb-ff10-4e23-9223-da3bd383bae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:42.715427 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28158eb-ff10-4e23-9223-da3bd383bae9 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:42.715621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0959a98-2b9c-46cf-a3af-e35f9fed0bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:42.715906 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28158eb-ff10-4e23-9223-da3bd383bae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:42.716235 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0959a98-2b9c-46cf-a3af-e35f9fed0bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:42.716358 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28158eb-ff10-4e23-9223-da3bd383bae9 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:42.716511 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0959a98-2b9c-46cf-a3af-e35f9fed0bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:42.716781 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0959a98-2b9c-46cf-a3af-e35f9fed0bb5 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:42.717339 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28158eb-ff10-4e23-9223-da3bd383bae9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:42.717527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0959a98-2b9c-46cf-a3af-e35f9fed0bb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:42.717826 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28158eb-ff10-4e23-9223-da3bd383bae9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:42.717933 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0959a98-2b9c-46cf-a3af-e35f9fed0bb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:42.719180 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0959a98-2b9c-46cf-a3af-e35f9fed0bb5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:42.719457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28158eb-ff10-4e23-9223-da3bd383bae9 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:42.728460 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 303/600] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:23:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:42.729451 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28158eb-ff10-4e23-9223-da3bd383bae9 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:42.732507 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 298/601] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:23:42.735165 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0959a98-2b9c-46cf-a3af-e35f9fed0bb5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:42.845436 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:42.857109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28158eb-ff10-4e23-9223-da3bd383bae9 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:42.861880 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:42.866724 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0959a98-2b9c-46cf-a3af-e35f9fed0bb5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:42.898628 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1764d6-a9a3-489a-b013-227519bc35a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:42.899479 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1764d6-a9a3-489a-b013-227519bc35a6 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:42.899858 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1764d6-a9a3-489a-b013-227519bc35a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:42.900277 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1764d6-a9a3-489a-b013-227519bc35a6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:42.901105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1764d6-a9a3-489a-b013-227519bc35a6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:42.902691 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1764d6-a9a3-489a-b013-227519bc35a6 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:42.910874 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12b98e-e7d1-43a2-9868-c03100984ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:42.911497 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12b98e-e7d1-43a2-9868-c03100984ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:42.911774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12b98e-e7d1-43a2-9868-c03100984ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:42.912045 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12b98e-e7d1-43a2-9868-c03100984ec4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:42.914168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-bb1764d6-a9a3-489a-b013-227519bc35a6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 13:23:42.914168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:42.914168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:23:42.914168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:23:42.914168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:23:42.914168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:23:42.914168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:23:42.914168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:42.914168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 13:23:42.914168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:42.914168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:23:42.914168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:42.914168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:42.914168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:42.914168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:42.914168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:42.916204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:42.916204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:42.916204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:42.916204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:42.916204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:42.916204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:42.916204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:42.916204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:23:42.916204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:23:42.916204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:23:42.916204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:23:42.916204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:42.916204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:42.916204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:42.916204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:42.916204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:42.916204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:42.918362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:42.918362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:42.918362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:42.918362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:42.918362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:42.918362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:42.918362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:42.918362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:42.918362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:42.918362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:42.918362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:23:42.918362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:42.918362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 13:23:42.918362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:42.918362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 304/602] 10.222.0.23 () {56 vars in 1132 bytes} [Thu Oct 20 13:23:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:42.919764 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1764d6-a9a3-489a-b013-227519bc35a6 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:42.929408 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12b98e-e7d1-43a2-9868-c03100984ec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:42.931317 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12b98e-e7d1-43a2-9868-c03100984ec4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:42.951477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-3a12b98e-e7d1-43a2-9868-c03100984ec4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:42.951477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:42.951477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:42.951477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:42.951477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:42.951477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:42.951477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:42.951477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:42.951477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:42.951477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:42.951477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:42.951477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:42.951477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:42.951477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:42.951477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:42.951477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:42.952714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:42.952714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:42.952714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:42.952714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:42.952714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:42.952714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:42.952714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:42.952714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:42.952714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:42.952714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:42.952714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:42.952714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:42.952714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:42.952714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:42.952714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:42.952714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:42.952714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:42.954275 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:42.954275 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:42.954275 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:42.954275 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 299/603] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:23:42.954275 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12b98e-e7d1-43a2-9868-c03100984ec4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:43.036404 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1764d6-a9a3-489a-b013-227519bc35a6 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:43.073013 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df5f92-d86a-4c8e-982a-378e1f4eefa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:43.074023 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df5f92-d86a-4c8e-982a-378e1f4eefa5 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:43.074501 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df5f92-d86a-4c8e-982a-378e1f4eefa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:43.074975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df5f92-d86a-4c8e-982a-378e1f4eefa5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:43.083602 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df5f92-d86a-4c8e-982a-378e1f4eefa5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:43.085206 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12b98e-e7d1-43a2-9868-c03100984ec4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:43.086110 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df5f92-d86a-4c8e-982a-378e1f4eefa5 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:43.098688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-46df5f92-d86a-4c8e-982a-378e1f4eefa5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 20 13:23:43.098688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:43.098688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:43.098688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:43.098688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:43.098688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:43.098688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:43.098688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:43.098688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:43.098688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:43.098688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:43.098688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:43.098688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:23:43.098688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:23:43.098688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:23:43.098688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:23:43.099649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:43.099649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:43.099649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:23:43.099649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:43.099649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:23:43.099649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:43.099649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:43.099649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:43.099649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:43.099649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:43.099649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:43.099649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:43.099649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:43.099649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:43.099649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:43.099649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:43.099649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:43.099649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:43.100617 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:43.100617 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:43.100617 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:23:43.100617 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:23:43.100617 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:23:43.100617 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:23:43.100617 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:23:43.100617 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:23:43.100617 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 20 13:23:43.100617 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:43.103149 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 305/604] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Oct 20 13:23:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:43.104115 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df5f92-d86a-4c8e-982a-378e1f4eefa5 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:43.130990 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8681f-95e5-4f49-8184-9eeb15042faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:43.131687 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8681f-95e5-4f49-8184-9eeb15042faf None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:43.132020 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8681f-95e5-4f49-8184-9eeb15042faf None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:43.132299 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8681f-95e5-4f49-8184-9eeb15042faf None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:43.132960 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c8681f-95e5-4f49-8184-9eeb15042faf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:43.133262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c8681f-95e5-4f49-8184-9eeb15042faf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:43.135163 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c8681f-95e5-4f49-8184-9eeb15042faf None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:43.137735 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:43.138176 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:43.145014 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 300/605] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Oct 20 13:23:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:43.147130 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8681f-95e5-4f49-8184-9eeb15042faf None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:43.217715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df5f92-d86a-4c8e-982a-378e1f4eefa5 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:43.260334 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f41d96-a651-4f07-8b91-c3ebdc4b151d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:43.261331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f41d96-a651-4f07-8b91-c3ebdc4b151d None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:43.261899 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f41d96-a651-4f07-8b91-c3ebdc4b151d None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:43.262647 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f41d96-a651-4f07-8b91-c3ebdc4b151d None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:43.263570 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f41d96-a651-4f07-8b91-c3ebdc4b151d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:43.264070 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f41d96-a651-4f07-8b91-c3ebdc4b151d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:43.265670 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f41d96-a651-4f07-8b91-c3ebdc4b151d None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:43.266799 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8681f-95e5-4f49-8184-9eeb15042faf None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:43.279378 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 306/606] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:43.280519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f41d96-a651-4f07-8b91-c3ebdc4b151d None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:43.314945 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f027073-24fb-40f3-8e99-bd951ef01a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:43.315807 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f027073-24fb-40f3-8e99-bd951ef01a07 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:43.316113 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f027073-24fb-40f3-8e99-bd951ef01a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:43.316414 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f027073-24fb-40f3-8e99-bd951ef01a07 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:43.317237 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f027073-24fb-40f3-8e99-bd951ef01a07 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:43.317546 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f027073-24fb-40f3-8e99-bd951ef01a07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:43.318978 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f027073-24fb-40f3-8e99-bd951ef01a07 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:43.343703 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 301/607] 10.222.0.23 () {56 vars in 1149 bytes} [Thu Oct 20 13:23:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:43.345618 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f027073-24fb-40f3-8e99-bd951ef01a07 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:43.381451 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:43.396828 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f41d96-a651-4f07-8b91-c3ebdc4b151d None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:43.428287 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:43.434454 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8061bd4-c92e-4c8f-aae5-1c299b3b9c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:43.435213 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8061bd4-c92e-4c8f-aae5-1c299b3b9c0b None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:43.435581 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8061bd4-c92e-4c8f-aae5-1c299b3b9c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:43.435904 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8061bd4-c92e-4c8f-aae5-1c299b3b9c0b None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:43.436640 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8061bd4-c92e-4c8f-aae5-1c299b3b9c0b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:43.437089 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8061bd4-c92e-4c8f-aae5-1c299b3b9c0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:43.438652 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8061bd4-c92e-4c8f-aae5-1c299b3b9c0b None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:43.452911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 307/608] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Oct 20 13:23:43 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:43.452911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8061bd4-c92e-4c8f-aae5-1c299b3b9c0b None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:43.513337 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f027073-24fb-40f3-8e99-bd951ef01a07 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:43.594960 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609f8f3-ada4-4fa2-b9c2-8761235f66e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:43.595621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609f8f3-ada4-4fa2-b9c2-8761235f66e7 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:43.595906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609f8f3-ada4-4fa2-b9c2-8761235f66e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:43.596180 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609f8f3-ada4-4fa2-b9c2-8761235f66e7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:43.610070 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609f8f3-ada4-4fa2-b9c2-8761235f66e7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:43.616229 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609f8f3-ada4-4fa2-b9c2-8761235f66e7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:43.636107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-1609f8f3-ada4-4fa2-b9c2-8761235f66e7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 20 13:23:43.636107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:43.636107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:43.636107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:43.636107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:43.636107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:43.636107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:43.636107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:43.636107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:43.636107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:43.636107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:43.636107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:43.636107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:23:43.636107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:23:43.636107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:23:43.636107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:23:43.637146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:43.637146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:43.637146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:23:43.637146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:43.637146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:23:43.637146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:43.637146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:43.637146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:43.637146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:43.637146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:43.637146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:43.637146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:43.637146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:43.637146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:43.637146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:43.637146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:43.637146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:43.637146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:43.638335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:43.638335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:43.638335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:23:43.638335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:23:43.638335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:23:43.638335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:23:43.638335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:23:43.638335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:23:43.638335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 20 13:23:43.638335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:43.641505 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 302/609] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 20 13:23:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:43.647473 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609f8f3-ada4-4fa2-b9c2-8761235f66e7 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:43.680627 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8061bd4-c92e-4c8f-aae5-1c299b3b9c0b None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:43.723826 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa53bd-2962-42bf-ba87-b0432900cec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:43.724449 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa53bd-2962-42bf-ba87-b0432900cec5 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:43.724733 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa53bd-2962-42bf-ba87-b0432900cec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:43.725057 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa53bd-2962-42bf-ba87-b0432900cec5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:43.753479 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:43.784650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609f8f3-ada4-4fa2-b9c2-8761235f66e7 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:43.802869 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa53bd-2962-42bf-ba87-b0432900cec5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:43.804615 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa53bd-2962-42bf-ba87-b0432900cec5 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:43.804996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:43.823806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f36b9-72ec-4009-b8f2-6e1a6c5463b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:43.824421 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f36b9-72ec-4009-b8f2-6e1a6c5463b4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:43.824696 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f36b9-72ec-4009-b8f2-6e1a6c5463b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:43.824996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f36b9-72ec-4009-b8f2-6e1a6c5463b4 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:43.825997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f36b9-72ec-4009-b8f2-6e1a6c5463b4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:43.826306 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f36b9-72ec-4009-b8f2-6e1a6c5463b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:43.828500 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f36b9-72ec-4009-b8f2-6e1a6c5463b4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:43.854154 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 303/610] 10.222.0.23 () {56 vars in 1135 bytes} [Thu Oct 20 13:23:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:23:43.854154 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7cb65-dd6f-4feb-b9c2-b359ef183a48 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:43.854154 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7cb65-dd6f-4feb-b9c2-b359ef183a48 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:43.854154 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7cb65-dd6f-4feb-b9c2-b359ef183a48 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:43.856982 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 304/611] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:43.859492 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69feff5f-4072-4e22-a0b1-0d9478ec3960 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:43.859785 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69feff5f-4072-4e22-a0b1-0d9478ec3960 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:43.860064 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69feff5f-4072-4e22-a0b1-0d9478ec3960 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:43.861533 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 305/612] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:43.862200 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69feff5f-4072-4e22-a0b1-0d9478ec3960 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:43.935047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 308/613] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:23:43 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 13:23:43.935047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1906681-3dbf-4329-ba34-6450b4c159cc None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:43.935047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1906681-3dbf-4329-ba34-6450b4c159cc None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:43.935047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1906681-3dbf-4329-ba34-6450b4c159cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:43.950945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-b1906681-3dbf-4329-ba34-6450b4c159cc None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:43.952087 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b1906681-3dbf-4329-ba34-6450b4c159cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:43.982267 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69feff5f-4072-4e22-a0b1-0d9478ec3960 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:43.999912 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:44.020909 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8c573-5201-48c3-acb5-49d7eedc7f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:44.021667 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8c573-5201-48c3-acb5-49d7eedc7f78 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:44.021945 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8c573-5201-48c3-acb5-49d7eedc7f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:44.022224 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8c573-5201-48c3-acb5-49d7eedc7f78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:44.024994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 309/614] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:23:44.026114 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8812e-4f70-4b7a-ad4a-ea9cf23d954a None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:44.026400 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8812e-4f70-4b7a-ad4a-ea9cf23d954a None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:44.026672 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8812e-4f70-4b7a-ad4a-ea9cf23d954a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:44.044529 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-d6f8812e-4f70-4b7a-ad4a-ea9cf23d954a None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:44.045650 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-d6f8812e-4f70-4b7a-ad4a-ea9cf23d954a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:44.099642 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:44.119755 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 310/615] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:23:44.120801 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8812e-4f70-4b7a-ad4a-ea9cf23d954a None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:44.126755 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8c573-5201-48c3-acb5-49d7eedc7f78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:44.128379 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8c573-5201-48c3-acb5-49d7eedc7f78 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:44.289877 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8812e-4f70-4b7a-ad4a-ea9cf23d954a None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:44.311341 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:44.313797 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 306/616] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:23:43 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 13:23:44.321164 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8c573-5201-48c3-acb5-49d7eedc7f78 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:44.374898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414770e4-b92f-41f0-97c3-458259f57429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:44.375405 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414770e4-b92f-41f0-97c3-458259f57429 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:44.375690 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414770e4-b92f-41f0-97c3-458259f57429 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:44.375966 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414770e4-b92f-41f0-97c3-458259f57429 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:44.388395 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414770e4-b92f-41f0-97c3-458259f57429 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:44.389966 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414770e4-b92f-41f0-97c3-458259f57429 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:44.407237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-414770e4-b92f-41f0-97c3-458259f57429 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:44.407237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:44.407237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:44.407237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:44.407237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:44.407237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:44.407237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:44.407237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:44.407237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:44.407237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:44.407237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:44.407237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:44.407237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:44.407237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:44.407237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:44.407237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:44.408214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:44.408214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:44.408214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:44.408214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:44.408214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:44.408214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:44.408214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:44.408214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:44.408214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:44.408214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:44.408214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:44.408214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:44.408214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:44.408214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:44.408214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:44.408214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:44.408214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:44.409116 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:44.409116 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:44.409116 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:44.417070 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 311/617] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:44.418069 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414770e4-b92f-41f0-97c3-458259f57429 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:44.439417 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:44.463794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8c573-5201-48c3-acb5-49d7eedc7f78 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:44.502315 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e01ea1-2ac3-4623-9bbb-dc9a4a565525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:44.502948 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e01ea1-2ac3-4623-9bbb-dc9a4a565525 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:44.503234 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e01ea1-2ac3-4623-9bbb-dc9a4a565525 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:44.504966 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e01ea1-2ac3-4623-9bbb-dc9a4a565525 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:44.520046 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e01ea1-2ac3-4623-9bbb-dc9a4a565525 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:44.521543 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e01ea1-2ac3-4623-9bbb-dc9a4a565525 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:44.527603 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414770e4-b92f-41f0-97c3-458259f57429 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:44.537195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-96e01ea1-2ac3-4623-9bbb-dc9a4a565525 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:44.537195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:44.537195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:44.537195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:44.537195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:44.537195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:44.537195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:44.537195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:44.537195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:44.537195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:44.537195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:44.537195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:44.537195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:44.537195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:44.537195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:44.537195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:44.538314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:44.538314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:44.538314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:44.538314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:44.538314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:44.538314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:44.538314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:44.538314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:44.538314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:44.538314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:44.538314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:44.538314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:44.538314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:44.538314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:44.538314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:44.538314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:44.538314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:44.539219 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:44.539219 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:44.539219 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:44.539219 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 307/618] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:44.539219 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e01ea1-2ac3-4623-9bbb-dc9a4a565525 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:44.568335 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9497f-b14a-46a9-9f7b-e8143c423ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:44.568914 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9497f-b14a-46a9-9f7b-e8143c423ba0 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:44.569311 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9497f-b14a-46a9-9f7b-e8143c423ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:44.569589 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9497f-b14a-46a9-9f7b-e8143c423ba0 None admin] PATH_INFO: `/v3/services/network` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:44.570171 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae9497f-b14a-46a9-9f7b-e8143c423ba0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:44.571532 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae9497f-b14a-46a9-9f7b-e8143c423ba0 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:44.581486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-7ae9497f-b14a-46a9-9f7b-e8143c423ba0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 13:23:44.581486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:44.581486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:23:44.581486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:23:44.581486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:23:44.581486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:23:44.581486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:23:44.581486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:44.581486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 13:23:44.581486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:44.581486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:23:44.581486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:44.581486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:44.581486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:44.581486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:44.581486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:44.581486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:44.583056 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:44.583056 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:44.583056 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:44.583056 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:44.583056 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:44.583056 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:44.583056 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:23:44.583056 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:23:44.583056 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:23:44.583056 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:23:44.583056 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:44.583056 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:44.583056 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:44.583056 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:44.583056 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:44.583056 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:44.583878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:44.583878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:44.583878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:44.583878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:44.583878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:44.583878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:44.583878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:44.583878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:44.583878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:44.583878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:44.583878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:23:44.583878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:44.583878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 20 13:23:44.583878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:44.583878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 312/619] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:44 2022] GET /identity/v3/services/network => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:44.585778 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9497f-b14a-46a9-9f7b-e8143c423ba0 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:44.615055 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:44.654149 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e01ea1-2ac3-4623-9bbb-dc9a4a565525 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:44.681702 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:44.690297 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565a190-8d5c-42c5-b946-053937cbe40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:44.690899 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565a190-8d5c-42c5-b946-053937cbe40a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:44.691460 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565a190-8d5c-42c5-b946-053937cbe40a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:44.691749 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565a190-8d5c-42c5-b946-053937cbe40a None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:44.692416 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b565a190-8d5c-42c5-b946-053937cbe40a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:44.692726 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b565a190-8d5c-42c5-b946-053937cbe40a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:44.693219 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9497f-b14a-46a9-9f7b-e8143c423ba0 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:44.694135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b565a190-8d5c-42c5-b946-053937cbe40a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:44.711914 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 308/620] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:44.711914 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565a190-8d5c-42c5-b946-053937cbe40a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:44.733675 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11745a-2d4a-41a1-8e94-ba3e442f0c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:44.734593 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11745a-2d4a-41a1-8e94-ba3e442f0c57 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:44.734869 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11745a-2d4a-41a1-8e94-ba3e442f0c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:44.735143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11745a-2d4a-41a1-8e94-ba3e442f0c57 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:44.735713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad11745a-2d4a-41a1-8e94-ba3e442f0c57 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:44.737109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad11745a-2d4a-41a1-8e94-ba3e442f0c57 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:44.748417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-ad11745a-2d4a-41a1-8e94-ba3e442f0c57 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 13:23:44.748417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:44.748417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:23:44.748417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:23:44.748417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:23:44.748417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:23:44.748417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:23:44.748417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:44.748417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 13:23:44.748417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:44.748417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:23:44.748417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:44.748417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:44.748417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:44.748417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:44.748417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:44.748417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:44.749437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:44.749437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:44.749437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:44.749437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:44.749437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:44.749437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:44.749437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:23:44.749437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:23:44.749437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:23:44.749437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:23:44.749437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:44.749437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:44.749437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:44.749437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:44.749437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:44.749437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:44.750280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:44.750280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:44.750280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:44.750280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:44.750280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:44.750280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:44.750280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:44.750280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:44.750280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:44.750280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:44.750280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:23:44.750280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:44.750280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 20 13:23:44.750280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:44.750280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 313/621] 10.222.0.23 () {56 vars in 1129 bytes} [Thu Oct 20 13:23:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:44.751062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11745a-2d4a-41a1-8e94-ba3e442f0c57 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:44.827820 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565a190-8d5c-42c5-b946-053937cbe40a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:44.844830 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:44.854210 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11745a-2d4a-41a1-8e94-ba3e442f0c57 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:44.866089 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7645c-f802-4625-8fc5-1bcabb528ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:44.866695 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7645c-f802-4625-8fc5-1bcabb528ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:44.866970 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7645c-f802-4625-8fc5-1bcabb528ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:44.867248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7645c-f802-4625-8fc5-1bcabb528ca9 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:44.867872 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7645c-f802-4625-8fc5-1bcabb528ca9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:44.868174 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7645c-f802-4625-8fc5-1bcabb528ca9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:44.869560 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7645c-f802-4625-8fc5-1bcabb528ca9 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:44.885992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 309/622] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:44.885992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7645c-f802-4625-8fc5-1bcabb528ca9 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:44.893033 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c99b3c-99c1-41c5-8575-fa42c0e3caf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:44.893638 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c99b3c-99c1-41c5-8575-fa42c0e3caf2 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:44.893905 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c99b3c-99c1-41c5-8575-fa42c0e3caf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:44.894713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c99b3c-99c1-41c5-8575-fa42c0e3caf2 None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:44.895348 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c99b3c-99c1-41c5-8575-fa42c0e3caf2 None admin] RBAC: Adding query filter params (name=network) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:44.895723 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c99b3c-99c1-41c5-8575-fa42c0e3caf2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:44.897096 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c99b3c-99c1-41c5-8575-fa42c0e3caf2 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:44.901449 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 314/623] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 20 13:23:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:44.902208 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c99b3c-99c1-41c5-8575-fa42c0e3caf2 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:44.941839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:44.995942 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7645c-f802-4625-8fc5-1bcabb528ca9 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:45.012067 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c99b3c-99c1-41c5-8575-fa42c0e3caf2 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:45.032073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb6a0b-ee32-42a5-a0af-87c53a81d3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:45.032705 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb6a0b-ee32-42a5-a0af-87c53a81d3da None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:45.032999 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb6a0b-ee32-42a5-a0af-87c53a81d3da None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:45.033272 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb6a0b-ee32-42a5-a0af-87c53a81d3da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:45.040520 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb6a0b-ee32-42a5-a0af-87c53a81d3da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:45.042000 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb6a0b-ee32-42a5-a0af-87c53a81d3da None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:45.046235 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b420a4-3a08-41b1-83f1-1243835b309a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:45.046819 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b420a4-3a08-41b1-83f1-1243835b309a None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:45.047097 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b420a4-3a08-41b1-83f1-1243835b309a None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:45.047365 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b420a4-3a08-41b1-83f1-1243835b309a None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:45.047979 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b420a4-3a08-41b1-83f1-1243835b309a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:45.048276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b420a4-3a08-41b1-83f1-1243835b309a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:45.049653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b420a4-3a08-41b1-83f1-1243835b309a None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:45.053278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-f6fb6a0b-ee32-42a5-a0af-87c53a81d3da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:45.053278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:45.053278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:45.053278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:45.053278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:45.053278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:45.053278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:45.053278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:45.053278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:45.053278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:45.053278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:45.053278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:45.053278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:23:45.053278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:23:45.053278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:23:45.054141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:23:45.054141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:45.054141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:45.054141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:45.054141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:45.054141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:45.054141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:45.054141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:45.054141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:45.054141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:45.054141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:45.054141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:45.054141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:45.054141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:45.054141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:45.054141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:45.054141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:45.055011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:23:45.055011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:23:45.055011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:23:45.055011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:23:45.055011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:23:45.055011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:23:45.055011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:45.055011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:45.055011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 310/624] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:45.055507 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb6a0b-ee32-42a5-a0af-87c53a81d3da None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:45.062984 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 315/625] 10.222.0.23 () {56 vars in 1124 bytes} [Thu Oct 20 13:23:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:45.063744 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b420a4-3a08-41b1-83f1-1243835b309a None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:45.088389 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:45.171228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb6a0b-ee32-42a5-a0af-87c53a81d3da None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:45.178067 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b420a4-3a08-41b1-83f1-1243835b309a None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:45.182754 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:45.211112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b8d00-ec94-41e3-9d95-7fa6c51b4fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:45.211734 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b8d00-ec94-41e3-9d95-7fa6c51b4fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:45.212014 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b8d00-ec94-41e3-9d95-7fa6c51b4fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:45.212286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b8d00-ec94-41e3-9d95-7fa6c51b4fe3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:45.213156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca301d-9ed3-4ba5-aee5-e8d29ffd9135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:45.213754 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca301d-9ed3-4ba5-aee5-e8d29ffd9135 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:45.214048 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca301d-9ed3-4ba5-aee5-e8d29ffd9135 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:45.214296 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca301d-9ed3-4ba5-aee5-e8d29ffd9135 None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:45.214919 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca301d-9ed3-4ba5-aee5-e8d29ffd9135 None admin] RBAC: Adding query filter params (type=network) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:45.215355 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca301d-9ed3-4ba5-aee5-e8d29ffd9135 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:45.216568 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca301d-9ed3-4ba5-aee5-e8d29ffd9135 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:45.219677 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b8d00-ec94-41e3-9d95-7fa6c51b4fe3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:45.221079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b8d00-ec94-41e3-9d95-7fa6c51b4fe3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:45.226613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 316/626] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 20 13:23:45 2022] GET /identity/v3/services?type=network => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:45.226613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca301d-9ed3-4ba5-aee5-e8d29ffd9135 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:45.238241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-6c6b8d00-ec94-41e3-9d95-7fa6c51b4fe3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:45.238241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:45.238241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:45.238241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:45.238241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:45.238241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:45.238241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:45.238241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:45.238241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:45.238241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:45.238241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:45.238241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:45.238241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:23:45.238241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:23:45.238241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:23:45.239255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:23:45.239255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:45.239255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:45.239255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:45.239255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:45.239255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:45.239255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:45.239255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:45.239255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:45.239255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:45.239255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:45.239255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:45.239255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:45.239255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:45.239255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:45.239255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:45.239255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:45.240184 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:23:45.240184 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:23:45.240184 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:23:45.240184 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:23:45.240184 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:23:45.240184 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:23:45.240184 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:45.240184 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:45.240184 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 311/627] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:45.240665 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b8d00-ec94-41e3-9d95-7fa6c51b4fe3 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:45.318561 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:45.335014 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca301d-9ed3-4ba5-aee5-e8d29ffd9135 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:45.348697 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b8d00-ec94-41e3-9d95-7fa6c51b4fe3 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:45.369387 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97c1bd-7a89-43cd-abda-9ea889b1ad24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:45.370078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97c1bd-7a89-43cd-abda-9ea889b1ad24 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:45.370424 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97c1bd-7a89-43cd-abda-9ea889b1ad24 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:45.370755 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97c1bd-7a89-43cd-abda-9ea889b1ad24 None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:45.371451 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f97c1bd-7a89-43cd-abda-9ea889b1ad24 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:45.371759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f97c1bd-7a89-43cd-abda-9ea889b1ad24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:45.373132 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f97c1bd-7a89-43cd-abda-9ea889b1ad24 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:45.378052 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 317/628] 10.222.0.23 () {56 vars in 1124 bytes} [Thu Oct 20 13:23:45 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:45.381706 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97c1bd-7a89-43cd-abda-9ea889b1ad24 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:45.387657 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47d4d0-92de-45f2-8159-da884ad1b5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:45.388364 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47d4d0-92de-45f2-8159-da884ad1b5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:45.388755 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47d4d0-92de-45f2-8159-da884ad1b5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:45.389143 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47d4d0-92de-45f2-8159-da884ad1b5e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:45.389929 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47d4d0-92de-45f2-8159-da884ad1b5e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:45.390331 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47d4d0-92de-45f2-8159-da884ad1b5e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:45.391842 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47d4d0-92de-45f2-8159-da884ad1b5e6 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:45.405935 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 312/629] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 20 13:23:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:45.407263 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47d4d0-92de-45f2-8159-da884ad1b5e6 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:45.439641 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:45.487548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97c1bd-7a89-43cd-abda-9ea889b1ad24 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:45.527270 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473f653-3093-428e-8af8-ac4600aab598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:45.527873 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473f653-3093-428e-8af8-ac4600aab598 None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:45.528150 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473f653-3093-428e-8af8-ac4600aab598 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:45.528417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473f653-3093-428e-8af8-ac4600aab598 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:45.529110 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473f653-3093-428e-8af8-ac4600aab598 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:45.530470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473f653-3093-428e-8af8-ac4600aab598 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:45.530693 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47d4d0-92de-45f2-8159-da884ad1b5e6 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:45.553147 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 318/630] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:23:45 2022] POST /identity/v3/endpoints => generated 337 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:23:45.554393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473f653-3093-428e-8af8-ac4600aab598 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:45.562592 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:45.595163 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d39b2b-10b4-4072-a5d6-9cf7abc2a677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:45.595953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d39b2b-10b4-4072-a5d6-9cf7abc2a677 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:45.597186 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d39b2b-10b4-4072-a5d6-9cf7abc2a677 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:45.597765 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d39b2b-10b4-4072-a5d6-9cf7abc2a677 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:45.599850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d39b2b-10b4-4072-a5d6-9cf7abc2a677 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:45.600404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d39b2b-10b4-4072-a5d6-9cf7abc2a677 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:45.602235 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d39b2b-10b4-4072-a5d6-9cf7abc2a677 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:45.621551 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 313/631] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 20 13:23:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:45.621551 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d39b2b-10b4-4072-a5d6-9cf7abc2a677 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:45.704254 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473f653-3093-428e-8af8-ac4600aab598 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:45.744351 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:45.754224 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b331bd43-2e20-4004-ab17-59d10bb63adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:45.755073 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331bd43-2e20-4004-ab17-59d10bb63adc None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:45.755160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331bd43-2e20-4004-ab17-59d10bb63adc None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:45.755390 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331bd43-2e20-4004-ab17-59d10bb63adc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:45.756059 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b331bd43-2e20-4004-ab17-59d10bb63adc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:45.757479 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b331bd43-2e20-4004-ab17-59d10bb63adc None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:45.758237 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d39b2b-10b4-4072-a5d6-9cf7abc2a677 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:45.780246 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 319/632] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:23:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:23:45.781186 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b331bd43-2e20-4004-ab17-59d10bb63adc None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:45.807016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdafdb-09d4-433d-b340-0e3725799f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:45.808048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdafdb-09d4-433d-b340-0e3725799f7f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:45.808048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdafdb-09d4-433d-b340-0e3725799f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:45.808216 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdafdb-09d4-433d-b340-0e3725799f7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:45.809107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbdafdb-09d4-433d-b340-0e3725799f7f None admin] RBAC: Adding query filter params (role.id=0f01a310371a4f6bbae7bb2e319a3b39, scope.project.id=4a81b6ecfb6e46a8b08c8bc2b4000fc7, user.id=85bb137d0fb74031a49a5c629b32c32b) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:45.809231 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbdafdb-09d4-433d-b340-0e3725799f7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:45.810653 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbdafdb-09d4-433d-b340-0e3725799f7f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:45.821527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 314/633] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:23:45 2022] GET /identity/v3/role_assignments?user.id=85bb137d0fb74031a49a5c629b32c32b&scope.project.id=4a81b6ecfb6e46a8b08c8bc2b4000fc7&role.id=0f01a310371a4f6bbae7bb2e319a3b39 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:45.821527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6323abe-5408-4984-9175-2e974a43825e None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:45.821527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6323abe-5408-4984-9175-2e974a43825e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:45.821527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6323abe-5408-4984-9175-2e974a43825e None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:45.822933 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 315/634] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:45.829221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087fc2b-7e4e-4e11-bcc0-338f311c9ab5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:45.829510 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087fc2b-7e4e-4e11-bcc0-338f311c9ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:45.829965 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087fc2b-7e4e-4e11-bcc0-338f311c9ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:45.857524 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a087fc2b-7e4e-4e11-bcc0-338f311c9ab5 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:45.859452 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a087fc2b-7e4e-4e11-bcc0-338f311c9ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:45.865230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:45.926389 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b331bd43-2e20-4004-ab17-59d10bb63adc None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:45.964534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 316/635] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 13:23:45.968977 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884d592-9be6-40e0-921b-b9a877f4922f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:45.968977 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884d592-9be6-40e0-921b-b9a877f4922f None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:45.968977 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884d592-9be6-40e0-921b-b9a877f4922f None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:45.968977 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884d592-9be6-40e0-921b-b9a877f4922f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:45.969381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884d592-9be6-40e0-921b-b9a877f4922f None admin] RBAC: Adding query filter params (role.id=2bcf35e1bc5f4634b0aa6411fad2f43f, scope.project.id=4a81b6ecfb6e46a8b08c8bc2b4000fc7, user.id=9834211488354651803000c7e8fab80b) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:45.973022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884d592-9be6-40e0-921b-b9a877f4922f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:45.973022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8884d592-9be6-40e0-921b-b9a877f4922f None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:45.979266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 320/636] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:23:45 2022] GET /identity/v3/role_assignments?user.id=9834211488354651803000c7e8fab80b&scope.project.id=4a81b6ecfb6e46a8b08c8bc2b4000fc7&role.id=2bcf35e1bc5f4634b0aa6411fad2f43f => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:46.251967 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a087fc2b-7e4e-4e11-bcc0-338f311c9ab5 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:46.334444 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:46.361704 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a087fc2b-7e4e-4e11-bcc0-338f311c9ab5 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:46.398859 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f6a03-fb14-4d7f-81b1-0ee38be4a019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:46.399472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f6a03-fb14-4d7f-81b1-0ee38be4a019 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:46.399750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f6a03-fb14-4d7f-81b1-0ee38be4a019 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:46.400022 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f6a03-fb14-4d7f-81b1-0ee38be4a019 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:46.400678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f6a03-fb14-4d7f-81b1-0ee38be4a019 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:46.402085 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f6a03-fb14-4d7f-81b1-0ee38be4a019 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:46.419793 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 317/637] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 20 13:23:46 2022] POST /identity/v3/services => generated 269 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:23:48.048333 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d525544-b8a7-4437-8b43-c58a68aec221 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:48.048929 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d525544-b8a7-4437-8b43-c58a68aec221 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:48.048929 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d525544-b8a7-4437-8b43-c58a68aec221 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:48.050993 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 321/638] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:48.056946 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232cef1-7dc8-4add-b193-b7c424d1dbb6 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:48.057448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232cef1-7dc8-4add-b193-b7c424d1dbb6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:48.057522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232cef1-7dc8-4add-b193-b7c424d1dbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:48.076229 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e232cef1-7dc8-4add-b193-b7c424d1dbb6 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:48.077626 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e232cef1-7dc8-4add-b193-b7c424d1dbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:48.123826 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ea121-b3b2-441a-a945-d5b102b6cb32 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:48.124454 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ea121-b3b2-441a-a945-d5b102b6cb32 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:48.124831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ea121-b3b2-441a-a945-d5b102b6cb32 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:48.126544 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 322/639] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:23:48.132296 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb131c8-f580-4178-92b5-d6944cd865ef None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:48.132707 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb131c8-f580-4178-92b5-d6944cd865ef None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:48.133122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb131c8-f580-4178-92b5-d6944cd865ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:48.152706 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-ddb131c8-f580-4178-92b5-d6944cd865ef None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:48.154209 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-ddb131c8-f580-4178-92b5-d6944cd865ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:48.160271 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 318/640] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:48 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:23:48.160271 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17d78c-b0a6-4031-8e3c-e5c91a8812ff None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:48.160271 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17d78c-b0a6-4031-8e3c-e5c91a8812ff None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:48.160271 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17d78c-b0a6-4031-8e3c-e5c91a8812ff None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:48.160271 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 319/641] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:48.165781 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b561e-5ea6-4f3c-9826-9758dab63fd5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:48.166067 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b561e-5ea6-4f3c-9826-9758dab63fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:48.166344 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b561e-5ea6-4f3c-9826-9758dab63fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:48.187436 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-256b561e-5ea6-4f3c-9826-9758dab63fd5 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:48.188723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-256b561e-5ea6-4f3c-9826-9758dab63fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:48.226042 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 323/642] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:48 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 13:23:48.250547 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:48.263597 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 320/643] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:48 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 13:23:48.435700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb131c8-f580-4178-92b5-d6944cd865ef None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:48.492477 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:48.513661 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b561e-5ea6-4f3c-9826-9758dab63fd5 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:48.546999 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb131c8-f580-4178-92b5-d6944cd865ef None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:48.584597 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b062b7-edf7-4054-91f1-e91ff0327191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:48.585256 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b062b7-edf7-4054-91f1-e91ff0327191 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:48.585533 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b062b7-edf7-4054-91f1-e91ff0327191 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:48.585807 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b062b7-edf7-4054-91f1-e91ff0327191 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:48.586390 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b062b7-edf7-4054-91f1-e91ff0327191 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:48.587768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b062b7-edf7-4054-91f1-e91ff0327191 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:48.597440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-67b062b7-edf7-4054-91f1-e91ff0327191 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 13:23:48.597440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:48.597440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:23:48.597440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:23:48.597440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:23:48.597440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:23:48.597440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:23:48.597440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:48.597440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 13:23:48.597440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:48.597440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:23:48.597440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:48.597440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:48.597440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:48.597440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:48.597440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:48.597440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:48.598538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:48.598538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:48.598538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:48.598538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:48.598538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:48.598538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:48.598538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:23:48.598538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:23:48.598538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:23:48.598538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:23:48.598538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:48.598538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:48.598538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:48.598538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:48.598538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:48.598538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:48.599452 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:48.599452 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:48.599452 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:48.599452 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:48.599452 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:48.599452 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:48.599452 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:48.599452 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:48.599452 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:48.599452 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:48.599452 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:23:48.599452 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:48.599452 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 13:23:48.599452 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:48.599452 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 324/644] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 20 13:23:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:48.600229 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b062b7-edf7-4054-91f1-e91ff0327191 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:48.631062 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b561e-5ea6-4f3c-9826-9758dab63fd5 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:48.670357 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fac2dc-ee44-4073-b598-e890426a43f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:48.671053 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fac2dc-ee44-4073-b598-e890426a43f3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:48.671270 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fac2dc-ee44-4073-b598-e890426a43f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:48.671536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fac2dc-ee44-4073-b598-e890426a43f3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:48.682346 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fac2dc-ee44-4073-b598-e890426a43f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:48.687207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-28fac2dc-ee44-4073-b598-e890426a43f3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 20 13:23:48.687207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:48.687207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:48.687207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:48.687207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:48.687207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:48.687207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:48.687207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:48.687207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:48.687207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:48.687207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:48.687207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:48.687207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:48.687207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:48.687207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:48.687207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:48.688269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:48.688269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:48.688269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:48.688269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:48.688269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:48.688269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:48.688269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:48.688269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:48.688269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:48.688269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:48.688269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:48.688269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:48.688269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:48.688269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:48.688269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:48.688269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:48.689464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:48.689464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:48.689464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:48.689464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:48.689464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:48.689464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:48.689464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:48.689464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:48.689464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:48.689464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:48.689464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 20 13:23:48.689464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:48.689464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 321/645] 10.222.0.23 () {56 vars in 1120 bytes} [Thu Oct 20 13:23:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:48.689464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fac2dc-ee44-4073-b598-e890426a43f3 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:48.715325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b062b7-edf7-4054-91f1-e91ff0327191 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:48.732019 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:48.742908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:48.749262 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec0e73-a736-4edb-b454-70d542dc8fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:48.749887 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec0e73-a736-4edb-b454-70d542dc8fca None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:48.750163 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec0e73-a736-4edb-b454-70d542dc8fca None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:48.750490 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec0e73-a736-4edb-b454-70d542dc8fca None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:48.751001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec0e73-a736-4edb-b454-70d542dc8fca None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:48.752364 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec0e73-a736-4edb-b454-70d542dc8fca None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:48.761572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-65ec0e73-a736-4edb-b454-70d542dc8fca None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 13:23:48.761572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:48.761572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:23:48.761572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:23:48.761572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:23:48.761572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:23:48.761572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:23:48.761572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:48.761572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 13:23:48.761572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:48.761572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:23:48.761572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:48.761572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:48.761572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:48.761572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:48.761572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:48.761572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:48.762898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:48.762898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:48.762898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:48.762898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:48.762898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:48.762898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:48.762898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:23:48.762898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:23:48.762898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:23:48.762898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:23:48.762898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:48.762898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:48.762898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:48.762898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:48.762898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:48.762898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:48.763737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:48.763737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:48.763737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:48.763737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:48.763737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:48.763737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:48.763737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:48.763737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:48.763737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:48.763737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:48.763737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:23:48.763737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:48.763737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 13:23:48.763737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:48.763737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 325/646] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Oct 20 13:23:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:48.764551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec0e73-a736-4edb-b454-70d542dc8fca None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:48.797176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fac2dc-ee44-4073-b598-e890426a43f3 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:48.830739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2c48e-4261-4d80-8936-0daddc69947d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:48.831346 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2c48e-4261-4d80-8936-0daddc69947d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:48.832681 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2c48e-4261-4d80-8936-0daddc69947d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:48.832992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2c48e-4261-4d80-8936-0daddc69947d None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:48.833623 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2c48e-4261-4d80-8936-0daddc69947d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:48.833928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2c48e-4261-4d80-8936-0daddc69947d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:48.835297 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2c48e-4261-4d80-8936-0daddc69947d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:48.839779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 322/647] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 20 13:23:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:48.841110 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e0aec-377d-4ff8-af59-81cd1b13642a None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:48.841389 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e0aec-377d-4ff8-af59-81cd1b13642a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:48.841660 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e0aec-377d-4ff8-af59-81cd1b13642a None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:48.843093 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 323/648] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:48.843791 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e0aec-377d-4ff8-af59-81cd1b13642a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:48.864489 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec0e73-a736-4edb-b454-70d542dc8fca None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:48.898217 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f1257-68fe-46f2-9f6a-f36c36ba01f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:48.898808 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f1257-68fe-46f2-9f6a-f36c36ba01f5 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:48.899084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f1257-68fe-46f2-9f6a-f36c36ba01f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:48.899352 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f1257-68fe-46f2-9f6a-f36c36ba01f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:48.900005 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f1257-68fe-46f2-9f6a-f36c36ba01f5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:48.900301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f1257-68fe-46f2-9f6a-f36c36ba01f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:48.901688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f1257-68fe-46f2-9f6a-f36c36ba01f5 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:48.909235 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 326/649] 10.222.0.23 () {56 vars in 1115 bytes} [Thu Oct 20 13:23:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:48.910712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f1257-68fe-46f2-9f6a-f36c36ba01f5 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:48.949684 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e0aec-377d-4ff8-af59-81cd1b13642a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:48.953470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:48.964945 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:48.986804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175a44c-4532-4df6-acd0-05a60b3ff0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:48.987404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175a44c-4532-4df6-acd0-05a60b3ff0b2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:48.987705 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175a44c-4532-4df6-acd0-05a60b3ff0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:48.989031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175a44c-4532-4df6-acd0-05a60b3ff0b2 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:48.989670 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175a44c-4532-4df6-acd0-05a60b3ff0b2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:48.989977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175a44c-4532-4df6-acd0-05a60b3ff0b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:48.991337 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175a44c-4532-4df6-acd0-05a60b3ff0b2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:48.995799 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 324/650] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Oct 20 13:23:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:48.996944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6794f-f763-4e19-8db1-17b34812e5be None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:48.997229 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6794f-f763-4e19-8db1-17b34812e5be None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:48.997501 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6794f-f763-4e19-8db1-17b34812e5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:49.012930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f1257-68fe-46f2-9f6a-f36c36ba01f5 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:49.015339 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-93e6794f-f763-4e19-8db1-17b34812e5be None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:49.016504 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-93e6794f-f763-4e19-8db1-17b34812e5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:49.046157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035e529-1d21-4686-b5df-5e1bd445a6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:49.046755 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035e529-1d21-4686-b5df-5e1bd445a6cd None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:49.047034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035e529-1d21-4686-b5df-5e1bd445a6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:49.047312 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035e529-1d21-4686-b5df-5e1bd445a6cd None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:49.047937 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7035e529-1d21-4686-b5df-5e1bd445a6cd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:49.048243 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7035e529-1d21-4686-b5df-5e1bd445a6cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:49.049655 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7035e529-1d21-4686-b5df-5e1bd445a6cd None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:49.054136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 327/651] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 20 13:23:48 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:49.054911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035e529-1d21-4686-b5df-5e1bd445a6cd None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:49.087897 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 325/652] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:48 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:23:49.088782 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6794f-f763-4e19-8db1-17b34812e5be None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:49.201104 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035e529-1d21-4686-b5df-5e1bd445a6cd None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:49.212582 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:49.243344 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6794f-f763-4e19-8db1-17b34812e5be None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:49.247061 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986851e-7b2e-4864-8dc4-6503f007c677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:49.247692 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986851e-7b2e-4864-8dc4-6503f007c677 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:49.247971 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986851e-7b2e-4864-8dc4-6503f007c677 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:49.248240 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986851e-7b2e-4864-8dc4-6503f007c677 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:49.254436 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:49.261314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986851e-7b2e-4864-8dc4-6503f007c677 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:49.263251 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986851e-7b2e-4864-8dc4-6503f007c677 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:49.279047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-c986851e-7b2e-4864-8dc4-6503f007c677 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:49.279047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:49.279047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:49.279047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:49.279047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:49.279047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:49.279047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:49.279047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:49.279047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:49.279047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:49.279047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:49.279047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:49.279047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:49.279047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:49.279047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:49.279047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:49.280022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:49.280022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:49.280022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:49.280022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:49.280022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:49.280022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:49.280022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:49.280022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:49.280022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:49.280022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:49.280022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:49.280022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:49.280022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:49.280022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:49.280022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:49.280022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:49.280022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:49.280919 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:49.280919 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:49.280919 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:49.280919 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 328/653] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:49.286752 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986851e-7b2e-4864-8dc4-6503f007c677 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:49.295827 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f97ec-d7d9-49ee-a4d9-ba9dd60618bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:49.296817 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f97ec-d7d9-49ee-a4d9-ba9dd60618bc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:49.297337 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f97ec-d7d9-49ee-a4d9-ba9dd60618bc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:49.297895 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f97ec-d7d9-49ee-a4d9-ba9dd60618bc None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:49.298721 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f97ec-d7d9-49ee-a4d9-ba9dd60618bc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:49.299311 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f97ec-d7d9-49ee-a4d9-ba9dd60618bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:49.301617 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f97ec-d7d9-49ee-a4d9-ba9dd60618bc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:49.307368 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 326/654] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Oct 20 13:23:49 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:49.382200 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f97ec-d7d9-49ee-a4d9-ba9dd60618bc None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:49.404774 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986851e-7b2e-4864-8dc4-6503f007c677 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:49.446153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf2e35-41e8-445e-92c0-51ed80196beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:49.446778 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf2e35-41e8-445e-92c0-51ed80196beb None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:49.447186 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf2e35-41e8-445e-92c0-51ed80196beb None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:49.447462 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf2e35-41e8-445e-92c0-51ed80196beb None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:49.448128 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf2e35-41e8-445e-92c0-51ed80196beb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:49.448442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf2e35-41e8-445e-92c0-51ed80196beb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:49.449909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf2e35-41e8-445e-92c0-51ed80196beb None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:49.464991 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 329/655] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:49.470178 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf2e35-41e8-445e-92c0-51ed80196beb None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:49.502703 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:49.518680 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f97ec-d7d9-49ee-a4d9-ba9dd60618bc None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:49.562843 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402d59a-f1e7-4ca2-bb4c-5f87189b324d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:49.563722 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402d59a-f1e7-4ca2-bb4c-5f87189b324d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:49.564619 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402d59a-f1e7-4ca2-bb4c-5f87189b324d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:49.565078 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402d59a-f1e7-4ca2-bb4c-5f87189b324d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:49.566413 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d402d59a-f1e7-4ca2-bb4c-5f87189b324d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:49.568949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d402d59a-f1e7-4ca2-bb4c-5f87189b324d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:49.580534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d402d59a-f1e7-4ca2-bb4c-5f87189b324d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 13:23:49.580534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:49.580534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:23:49.580534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:23:49.580534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:23:49.580534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:23:49.580534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:23:49.580534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:49.580534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 13:23:49.580534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:49.580534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:23:49.580534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:49.580534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:49.580534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:49.580534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:49.580534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:49.581845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:49.581845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:49.581845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:49.581845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:49.581845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:49.581845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:49.581845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:49.581845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:23:49.581845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:23:49.581845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:23:49.581845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:23:49.581845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:49.581845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:49.581845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:49.581845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:49.581845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:49.581845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:49.583135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:49.583135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:49.583135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:49.583135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:49.583135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:49.583135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:49.583135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:49.583135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:49.583135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:49.583135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:49.583135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:23:49.583135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:49.583135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 13:23:49.583135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:49.585779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 327/656] 10.222.0.23 () {56 vars in 1132 bytes} [Thu Oct 20 13:23:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:23:49.592308 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402d59a-f1e7-4ca2-bb4c-5f87189b324d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:49.598283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf2e35-41e8-445e-92c0-51ed80196beb None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:49.635421 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:49.647503 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa30efb-e884-4444-b99d-fdfe74398e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:49.649205 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa30efb-e884-4444-b99d-fdfe74398e48 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:49.649722 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa30efb-e884-4444-b99d-fdfe74398e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:49.650461 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa30efb-e884-4444-b99d-fdfe74398e48 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:49.661459 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa30efb-e884-4444-b99d-fdfe74398e48 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:49.664111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa30efb-e884-4444-b99d-fdfe74398e48 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:49.680194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-daa30efb-e884-4444-b99d-fdfe74398e48 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 20 13:23:49.680194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:49.680194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:49.680194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:49.680194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:49.680194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:49.680194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:49.680194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:49.680194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:49.680194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:49.680194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:49.680194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:49.680194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:23:49.680194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:23:49.680194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:23:49.680194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:23:49.681211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:49.681211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:49.681211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:23:49.681211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:49.681211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:23:49.681211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:49.681211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:49.681211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:49.681211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:49.681211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:49.681211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:49.681211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:49.681211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:49.681211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:49.681211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:49.681211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:49.681211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:49.681211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:49.682125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:49.682125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:49.682125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:23:49.682125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:23:49.682125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:23:49.682125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:23:49.682125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:23:49.682125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:23:49.682125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 20 13:23:49.682125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:49.685242 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 330/657] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Oct 20 13:23:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:23:49.690351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa30efb-e884-4444-b99d-fdfe74398e48 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:49.710632 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402d59a-f1e7-4ca2-bb4c-5f87189b324d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:49.749445 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb53f52-1767-4c77-aa00-51c4c6ca70f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:49.750835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb53f52-1767-4c77-aa00-51c4c6ca70f4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:49.751324 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb53f52-1767-4c77-aa00-51c4c6ca70f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:49.751799 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb53f52-1767-4c77-aa00-51c4c6ca70f4 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:49.753170 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb53f52-1767-4c77-aa00-51c4c6ca70f4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:49.754310 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb53f52-1767-4c77-aa00-51c4c6ca70f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:49.756199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb53f52-1767-4c77-aa00-51c4c6ca70f4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:49.768576 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 328/658] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Oct 20 13:23:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:49.770303 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb53f52-1767-4c77-aa00-51c4c6ca70f4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:49.791371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:49.814368 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa30efb-e884-4444-b99d-fdfe74398e48 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:49.856142 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d916a-8bd4-424d-822e-29cab374d106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:49.857059 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d916a-8bd4-424d-822e-29cab374d106 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:49.858231 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d916a-8bd4-424d-822e-29cab374d106 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:49.858522 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d916a-8bd4-424d-822e-29cab374d106 None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:49.859341 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d916a-8bd4-424d-822e-29cab374d106 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:49.859657 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d916a-8bd4-424d-822e-29cab374d106 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:49.861149 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d916a-8bd4-424d-822e-29cab374d106 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:49.886158 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 331/659] 10.222.0.23 () {56 vars in 1149 bytes} [Thu Oct 20 13:23:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:23:49.892354 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d916a-8bd4-424d-822e-29cab374d106 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:49.895361 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:49.898595 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb53f52-1767-4c77-aa00-51c4c6ca70f4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:49.937100 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4579e0-d705-4e04-9daa-1c4b2f160708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:49.937761 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4579e0-d705-4e04-9daa-1c4b2f160708 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:49.938042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4579e0-d705-4e04-9daa-1c4b2f160708 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:49.938315 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4579e0-d705-4e04-9daa-1c4b2f160708 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:49.938950 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4579e0-d705-4e04-9daa-1c4b2f160708 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:49.939255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4579e0-d705-4e04-9daa-1c4b2f160708 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:49.940621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4579e0-d705-4e04-9daa-1c4b2f160708 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:49.946853 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 329/660] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Oct 20 13:23:49 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:49.954537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4579e0-d705-4e04-9daa-1c4b2f160708 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:50.025036 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d916a-8bd4-424d-822e-29cab374d106 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:50.070492 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b8339-bcda-4a47-a493-0c55e43acb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:50.071122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b8339-bcda-4a47-a493-0c55e43acb47 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:50.071446 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b8339-bcda-4a47-a493-0c55e43acb47 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:50.071733 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b8339-bcda-4a47-a493-0c55e43acb47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:50.082269 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:50.086195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4579e0-d705-4e04-9daa-1c4b2f160708 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:50.087714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b8339-bcda-4a47-a493-0c55e43acb47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:50.089699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b8339-bcda-4a47-a493-0c55e43acb47 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:50.109398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-776b8339-bcda-4a47-a493-0c55e43acb47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:50.109398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:50.109398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:50.109398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:50.109398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:50.109398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:50.109398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:50.109398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:50.109398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:50.109398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:50.109398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:50.109398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:50.109398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:50.109398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:50.109398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:50.109398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:50.110385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:50.110385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:50.110385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:50.110385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:50.110385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:50.110385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:50.110385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:50.110385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:50.110385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:50.110385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:50.110385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:50.110385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:50.110385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:50.110385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:50.110385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:50.110385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:50.110385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:50.111246 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:50.111246 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:50.111246 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:50.114233 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 332/661] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:23:50.119371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b8339-bcda-4a47-a493-0c55e43acb47 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:50.133758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46674ee-4e32-455f-9f2d-563d67fcf2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:50.135042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46674ee-4e32-455f-9f2d-563d67fcf2d3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:50.135540 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46674ee-4e32-455f-9f2d-563d67fcf2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:50.136016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46674ee-4e32-455f-9f2d-563d67fcf2d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:50.136946 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46674ee-4e32-455f-9f2d-563d67fcf2d3 None admin] RBAC: Adding query filter params (interface=public, service_id=71cc53355fc34f8c950809683122c819, region_id=RegionOne) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:50.137713 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46674ee-4e32-455f-9f2d-563d67fcf2d3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:50.139741 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46674ee-4e32-455f-9f2d-563d67fcf2d3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:50.151174 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 330/662] 10.222.0.23 () {56 vars in 1251 bytes} [Thu Oct 20 13:23:49 2022] GET /identity/v3/endpoints?service_id=71cc53355fc34f8c950809683122c819&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:50.151989 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46674ee-4e32-455f-9f2d-563d67fcf2d3 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:50.186557 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:50.232378 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b8339-bcda-4a47-a493-0c55e43acb47 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:50.269828 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46674ee-4e32-455f-9f2d-563d67fcf2d3 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:50.270186 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60680095-711a-447f-affb-2cbe3330a582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:50.270986 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60680095-711a-447f-affb-2cbe3330a582 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:50.271410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60680095-711a-447f-affb-2cbe3330a582 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:50.271901 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60680095-711a-447f-affb-2cbe3330a582 None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:50.272803 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60680095-711a-447f-affb-2cbe3330a582 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:50.273383 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60680095-711a-447f-affb-2cbe3330a582 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:50.274998 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60680095-711a-447f-affb-2cbe3330a582 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:50.295480 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 333/663] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:23:50.296716 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60680095-711a-447f-affb-2cbe3330a582 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:50.311506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538d0e9-e244-4a77-b4f0-a94d57cbe625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:50.312286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538d0e9-e244-4a77-b4f0-a94d57cbe625 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:50.312671 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538d0e9-e244-4a77-b4f0-a94d57cbe625 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:50.313075 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538d0e9-e244-4a77-b4f0-a94d57cbe625 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:50.313812 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538d0e9-e244-4a77-b4f0-a94d57cbe625 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:50.315412 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538d0e9-e244-4a77-b4f0-a94d57cbe625 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:50.324970 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 331/664] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Oct 20 13:23:50 2022] GET /identity/v3/services => generated 1029 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:23:50.367343 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:50.427940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60680095-711a-447f-affb-2cbe3330a582 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:50.466660 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467ec00-b5be-43d2-b06c-445d2ed8f3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:50.467619 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467ec00-b5be-43d2-b06c-445d2ed8f3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:50.468042 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467ec00-b5be-43d2-b06c-445d2ed8f3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:50.468494 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467ec00-b5be-43d2-b06c-445d2ed8f3f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:50.477713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467ec00-b5be-43d2-b06c-445d2ed8f3f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:50.479437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467ec00-b5be-43d2-b06c-445d2ed8f3f5 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:50.492997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-2467ec00-b5be-43d2-b06c-445d2ed8f3f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:50.492997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:50.492997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:50.492997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:50.492997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:50.492997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:50.492997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:50.492997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:50.492997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:50.492997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:50.492997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:50.492997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:50.492997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:23:50.492997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:23:50.492997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:23:50.494117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:23:50.494117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:50.494117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:50.494117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:50.494117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:50.494117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:50.494117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:50.494117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:50.494117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:50.494117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:50.494117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:50.494117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:50.494117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:50.494117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:50.494117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:50.494117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:50.494117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:50.495175 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:23:50.495175 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:23:50.495175 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:23:50.495175 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:23:50.495175 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:23:50.495175 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:23:50.495175 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:50.495175 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:50.497295 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 334/665] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:50.503232 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538d0e9-e244-4a77-b4f0-a94d57cbe625 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:50.604816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:50.615612 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538d0e9-e244-4a77-b4f0-a94d57cbe625 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:50.651309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701a046-d795-4423-bdcd-abbc0340e2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:50.651975 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701a046-d795-4423-bdcd-abbc0340e2f2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:50.652255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701a046-d795-4423-bdcd-abbc0340e2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:50.652527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701a046-d795-4423-bdcd-abbc0340e2f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:50.653314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6701a046-d795-4423-bdcd-abbc0340e2f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:50.653704 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6701a046-d795-4423-bdcd-abbc0340e2f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:50.655100 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6701a046-d795-4423-bdcd-abbc0340e2f2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:50.668580 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 332/666] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 20 13:23:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:50.674463 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467ec00-b5be-43d2-b06c-445d2ed8f3f5 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:50.783296 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467ec00-b5be-43d2-b06c-445d2ed8f3f5 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:50.794934 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:50.822116 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f747bac-fee4-434b-a7e8-7d58636c3083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:50.822776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f747bac-fee4-434b-a7e8-7d58636c3083 None admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:50.823054 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f747bac-fee4-434b-a7e8-7d58636c3083 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:50.823329 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f747bac-fee4-434b-a7e8-7d58636c3083 None admin] PATH_INFO: `/v3/projects/4a81b6ecfb6e46a8b08c8bc2b4000fc7/users/85bb137d0fb74031a49a5c629b32c32b/roles/ae10ab7a1f7f4eddae7ca60cc47d60d1` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:50.825485 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f747bac-fee4-434b-a7e8-7d58636c3083 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a81b6ecfb6e46a8b08c8bc2b4000fc7, user_id=85bb137d0fb74031a49a5c629b32c32b, role_id=ae10ab7a1f7f4eddae7ca60cc47d60d1)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:50.826923 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f747bac-fee4-434b-a7e8-7d58636c3083 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:50.842739 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 335/667] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:23:50 2022] PUT /identity/v3/projects/4a81b6ecfb6e46a8b08c8bc2b4000fc7/users/85bb137d0fb74031a49a5c629b32c32b/roles/ae10ab7a1f7f4eddae7ca60cc47d60d1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:23:51.295830 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2b4c5-1d4a-4d2d-a0d3-541cf78e3935 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:51.296357 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2b4c5-1d4a-4d2d-a0d3-541cf78e3935 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:51.296357 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2b4c5-1d4a-4d2d-a0d3-541cf78e3935 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:51.297947 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 333/668] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:51.303434 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965ac9b-9139-4bff-ba77-317624f574e1 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:51.303892 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965ac9b-9139-4bff-ba77-317624f574e1 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:51.303978 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965ac9b-9139-4bff-ba77-317624f574e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:51.325154 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-1965ac9b-9139-4bff-ba77-317624f574e1 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:51.326367 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-1965ac9b-9139-4bff-ba77-317624f574e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:51.396970 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 336/669] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:51 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:23:51.635176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac26033-9686-4af7-9cdb-c0b6bd50fefb None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:51.635526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac26033-9686-4af7-9cdb-c0b6bd50fefb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:51.635731 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac26033-9686-4af7-9cdb-c0b6bd50fefb None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:51.637672 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 334/670] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:51.643013 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461f0c0-f304-42e7-8b0a-81da106b9258 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:51.643307 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461f0c0-f304-42e7-8b0a-81da106b9258 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:51.643586 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461f0c0-f304-42e7-8b0a-81da106b9258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:51.663774 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-8461f0c0-f304-42e7-8b0a-81da106b9258 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:51.664991 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-8461f0c0-f304-42e7-8b0a-81da106b9258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:51.684085 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac26033-9686-4af7-9cdb-c0b6bd50fefb None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:51.743565 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 337/671] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:51 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:23:51.806929 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac26033-9686-4af7-9cdb-c0b6bd50fefb None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:51.842618 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6357e2-581c-4513-99f3-ad4c44f57b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:51.843226 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6357e2-581c-4513-99f3-ad4c44f57b19 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:51.843509 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6357e2-581c-4513-99f3-ad4c44f57b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:51.843786 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6357e2-581c-4513-99f3-ad4c44f57b19 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:51.844563 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6357e2-581c-4513-99f3-ad4c44f57b19 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:51.845959 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6357e2-581c-4513-99f3-ad4c44f57b19 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:51.862246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 335/672] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 20 13:23:51 2022] POST /identity/v3/services => generated 251 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:23:52.037325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461f0c0-f304-42e7-8b0a-81da106b9258 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:52.069901 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:52.147168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461f0c0-f304-42e7-8b0a-81da106b9258 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:52.189101 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cfe3a3-c80c-4ed1-87e7-d8ae7598bc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:52.189711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfe3a3-c80c-4ed1-87e7-d8ae7598bc29 None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:52.189990 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfe3a3-c80c-4ed1-87e7-d8ae7598bc29 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:52.190263 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfe3a3-c80c-4ed1-87e7-d8ae7598bc29 None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:52.190968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cfe3a3-c80c-4ed1-87e7-d8ae7598bc29 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:52.193072 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cfe3a3-c80c-4ed1-87e7-d8ae7598bc29 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:52.217036 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 338/673] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 20 13:23:52 2022] POST /identity/v3/services => generated 241 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:23:52.696732 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd63a9c-afed-44a6-a1ad-26636f917b9d None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:52.697813 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd63a9c-afed-44a6-a1ad-26636f917b9d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:52.698315 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd63a9c-afed-44a6-a1ad-26636f917b9d None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:52.700161 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 336/674] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:52.705798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3873e-b435-44e7-a7d8-fc518da9efd0 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:52.707327 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3873e-b435-44e7-a7d8-fc518da9efd0 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:52.707813 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3873e-b435-44e7-a7d8-fc518da9efd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:52.727625 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-0dd3873e-b435-44e7-a7d8-fc518da9efd0 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:52.729152 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-0dd3873e-b435-44e7-a7d8-fc518da9efd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:52.804574 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 339/675] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:23:53.046486 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffc435-cacb-41da-a190-672705d6eaa9 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:53.046851 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffc435-cacb-41da-a190-672705d6eaa9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:53.047087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffc435-cacb-41da-a190-672705d6eaa9 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:53.048627 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 337/676] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:53.054346 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcbbca-d056-408d-8d7c-27680da78eb2 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:53.054692 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcbbca-d056-408d-8d7c-27680da78eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:53.054910 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcbbca-d056-408d-8d7c-27680da78eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:53.073103 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-5bdcbbca-d056-408d-8d7c-27680da78eb2 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:53.074267 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-5bdcbbca-d056-408d-8d7c-27680da78eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:53.082634 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:53.098650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffc435-cacb-41da-a190-672705d6eaa9 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:53.118341 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:53.148086 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 340/677] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:23:53.224570 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffc435-cacb-41da-a190-672705d6eaa9 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:53.259327 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50c241-a64f-4802-9cd4-04cdb39119fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:53.259961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50c241-a64f-4802-9cd4-04cdb39119fa None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:53.260244 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50c241-a64f-4802-9cd4-04cdb39119fa None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:53.260519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50c241-a64f-4802-9cd4-04cdb39119fa None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:53.261199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50c241-a64f-4802-9cd4-04cdb39119fa None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:53.262881 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50c241-a64f-4802-9cd4-04cdb39119fa None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:53.272506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-fa50c241-a64f-4802-9cd4-04cdb39119fa None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 13:23:53.272506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:53.272506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:23:53.272506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:23:53.272506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:23:53.272506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:23:53.272506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:23:53.272506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:53.272506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 13:23:53.272506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:53.272506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:23:53.272506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:53.272506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:53.272506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:53.272506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:53.272506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:53.273495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:53.273495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:53.273495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:53.273495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:53.273495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:53.273495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:53.273495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:53.273495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:23:53.273495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:23:53.273495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:23:53.273495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:23:53.273495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:53.273495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:53.273495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:53.273495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:53.273495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:53.273495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:53.274566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:53.274566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:53.274566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:53.274566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:53.274566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:53.274566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:53.274566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:53.274566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:53.274566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:53.274566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:53.274566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:23:53.274566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:53.274566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 20 13:23:53.274566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:53.274566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 338/678] 10.222.0.23 () {56 vars in 1132 bytes} [Thu Oct 20 13:23:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:53.281776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdcbbca-d056-408d-8d7c-27680da78eb2 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:53.386289 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdcbbca-d056-408d-8d7c-27680da78eb2 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:53.416470 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50c241-a64f-4802-9cd4-04cdb39119fa None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:53.424090 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57858bb0-0679-45b2-a2a3-d0c8a1cd2c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:53.424695 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57858bb0-0679-45b2-a2a3-d0c8a1cd2c08 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:53.425132 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57858bb0-0679-45b2-a2a3-d0c8a1cd2c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:53.425408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57858bb0-0679-45b2-a2a3-d0c8a1cd2c08 None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:53.426181 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57858bb0-0679-45b2-a2a3-d0c8a1cd2c08 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:53.426483 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57858bb0-0679-45b2-a2a3-d0c8a1cd2c08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:53.427842 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57858bb0-0679-45b2-a2a3-d0c8a1cd2c08 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:53.432789 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 341/679] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Oct 20 13:23:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:53.438101 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57858bb0-0679-45b2-a2a3-d0c8a1cd2c08 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:53.447778 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:53.509914 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:53.537204 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50c241-a64f-4802-9cd4-04cdb39119fa None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:53.569587 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57858bb0-0679-45b2-a2a3-d0c8a1cd2c08 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:53.576558 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e01bf6-c3cb-493e-ad9f-895da35d2098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:53.577225 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e01bf6-c3cb-493e-ad9f-895da35d2098 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:53.577941 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e01bf6-c3cb-493e-ad9f-895da35d2098 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:53.577941 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e01bf6-c3cb-493e-ad9f-895da35d2098 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:53.589749 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e01bf6-c3cb-493e-ad9f-895da35d2098 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:23:53.594995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-c2e01bf6-c3cb-493e-ad9f-895da35d2098 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 20 13:23:53.594995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:53.594995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:53.594995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:53.594995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:53.594995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:53.594995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:53.594995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:53.594995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:53.594995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:53.594995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:53.594995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:53.594995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:23:53.594995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:23:53.594995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:23:53.594995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:23:53.596105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:23:53.596105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:23:53.596105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:23:53.596105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:23:53.596105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:53.596105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:53.596105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:53.596105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:53.596105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:53.596105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:53.596105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:53.596105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:53.596105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:53.596105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:53.596105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:53.596105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:53.598458 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:53.598458 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:53.598458 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:53.598458 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:53.598458 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:23:53.598458 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:23:53.598458 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:23:53.598458 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:23:53.598458 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:23:53.598458 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:23:53.598458 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 20 13:23:53.598458 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:53.598458 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 339/680] 10.222.0.23 () {56 vars in 1120 bytes} [Thu Oct 20 13:23:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:23:53.602362 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e01bf6-c3cb-493e-ad9f-895da35d2098 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:53.620717 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7bd8f-ed0d-4447-804e-7c307da0c952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:53.621889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7bd8f-ed0d-4447-804e-7c307da0c952 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:53.622402 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7bd8f-ed0d-4447-804e-7c307da0c952 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:53.622917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7bd8f-ed0d-4447-804e-7c307da0c952 None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:53.623838 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7bd8f-ed0d-4447-804e-7c307da0c952 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:53.624341 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7bd8f-ed0d-4447-804e-7c307da0c952 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:53.626157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7bd8f-ed0d-4447-804e-7c307da0c952 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:53.632124 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 342/681] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Oct 20 13:23:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:23:53.640786 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7bd8f-ed0d-4447-804e-7c307da0c952 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:53.712339 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e01bf6-c3cb-493e-ad9f-895da35d2098 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:53.722020 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:53.754587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f457b2a-f75c-46d8-bf2a-81f8e08911d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:53.755198 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f457b2a-f75c-46d8-bf2a-81f8e08911d8 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:53.755481 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f457b2a-f75c-46d8-bf2a-81f8e08911d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:53.755759 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f457b2a-f75c-46d8-bf2a-81f8e08911d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:53.756147 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7bd8f-ed0d-4447-804e-7c307da0c952 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:53.756425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f457b2a-f75c-46d8-bf2a-81f8e08911d8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:53.756734 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f457b2a-f75c-46d8-bf2a-81f8e08911d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:53.758312 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f457b2a-f75c-46d8-bf2a-81f8e08911d8 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:53.761770 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:53.771229 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 340/682] 10.222.0.23 () {56 vars in 1115 bytes} [Thu Oct 20 13:23:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:23:53.777885 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f457b2a-f75c-46d8-bf2a-81f8e08911d8 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:53.805522 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4c7c7-f4bc-4cb1-b2ef-ee4aaaf9ac19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:53.807162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4c7c7-f4bc-4cb1-b2ef-ee4aaaf9ac19 None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:53.807641 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4c7c7-f4bc-4cb1-b2ef-ee4aaaf9ac19 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:53.808115 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4c7c7-f4bc-4cb1-b2ef-ee4aaaf9ac19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:53.809576 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e4c7c7-f4bc-4cb1-b2ef-ee4aaaf9ac19 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:53.811653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e4c7c7-f4bc-4cb1-b2ef-ee4aaaf9ac19 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:53.849136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 343/683] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:23:53 2022] POST /identity/v3/endpoints => generated 347 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:23:53.901599 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f457b2a-f75c-46d8-bf2a-81f8e08911d8 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:53.950660 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7edc6-b836-48f3-9d70-f6fed459d05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:53.951300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7edc6-b836-48f3-9d70-f6fed459d05f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:53.951625 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7edc6-b836-48f3-9d70-f6fed459d05f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:53.951899 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7edc6-b836-48f3-9d70-f6fed459d05f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:53.964029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7edc6-b836-48f3-9d70-f6fed459d05f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:53.965619 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7edc6-b836-48f3-9d70-f6fed459d05f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:53.982723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-e3c7edc6-b836-48f3-9d70-f6fed459d05f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:53.982723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:53.982723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:53.982723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:53.982723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:53.982723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:53.982723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:53.982723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:53.982723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:53.982723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:53.982723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:53.982723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:53.982723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:53.982723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:53.982723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:53.982723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:53.983727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:53.983727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:53.983727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:53.983727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:53.983727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:53.983727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:53.983727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:53.983727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:53.983727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:53.983727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:53.983727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:53.983727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:53.983727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:53.983727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:53.983727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:53.983727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:53.983727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:53.985313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:53.985313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:53.985313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:53.985313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 341/684] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:53.989912 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4c7c7-f4bc-4cb1-b2ef-ee4aaaf9ac19 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:54.108978 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4c7c7-f4bc-4cb1-b2ef-ee4aaaf9ac19 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:54.138807 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:54.149244 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef48b9-8ce6-4211-aef0-f53d6f2ad4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:54.150077 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef48b9-8ce6-4211-aef0-f53d6f2ad4ac None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:54.150785 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef48b9-8ce6-4211-aef0-f53d6f2ad4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:54.151065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef48b9-8ce6-4211-aef0-f53d6f2ad4ac None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:54.151787 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef48b9-8ce6-4211-aef0-f53d6f2ad4ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:54.152031 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef48b9-8ce6-4211-aef0-f53d6f2ad4ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:54.153789 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef48b9-8ce6-4211-aef0-f53d6f2ad4ac None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:54.171343 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 344/685] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:54.176892 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7edc6-b836-48f3-9d70-f6fed459d05f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:54.227476 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:54.275142 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e67afc-aef6-421c-ab61-1372180a61bd None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:54.275443 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e67afc-aef6-421c-ab61-1372180a61bd None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:54.276338 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e67afc-aef6-421c-ab61-1372180a61bd None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:54.278380 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 345/686] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:54.285496 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9141b0f-8c7a-4a57-989a-060ca1cc76aa None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:54.286907 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9141b0f-8c7a-4a57-989a-060ca1cc76aa None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:54.287442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9141b0f-8c7a-4a57-989a-060ca1cc76aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:54.291419 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7edc6-b836-48f3-9d70-f6fed459d05f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:54.308363 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-e9141b0f-8c7a-4a57-989a-060ca1cc76aa None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:54.310002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-e9141b0f-8c7a-4a57-989a-060ca1cc76aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:54.332508 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61106f-5ca5-45c7-8f97-129951da06a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:54.333194 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61106f-5ca5-45c7-8f97-129951da06a2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:54.333476 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61106f-5ca5-45c7-8f97-129951da06a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:54.333851 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61106f-5ca5-45c7-8f97-129951da06a2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:54.346620 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61106f-5ca5-45c7-8f97-129951da06a2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:54.348842 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61106f-5ca5-45c7-8f97-129951da06a2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:54.361137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-3b61106f-5ca5-45c7-8f97-129951da06a2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 20 13:23:54.361137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:54.361137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:54.361137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:54.361137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:54.361137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:54.361137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:54.361137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:54.361137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:54.361137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:54.361137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:54.361137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:54.361137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:23:54.361137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:23:54.361137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:23:54.361137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:23:54.362114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:54.362114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:54.362114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:23:54.362114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:54.362114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:23:54.362114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:23:54.362114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:54.362114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:54.362114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:54.362114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:54.362114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:54.362114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:54.362114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:54.362114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:54.362114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:54.362114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:54.362114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:54.362114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:54.363101 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:54.363101 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:54.363101 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:23:54.363101 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:23:54.363101 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:23:54.363101 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:23:54.363101 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:23:54.363101 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:23:54.363101 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 20 13:23:54.363101 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:54.367231 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 342/687] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Oct 20 13:23:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:54.368591 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61106f-5ca5-45c7-8f97-129951da06a2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:54.413568 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 346/688] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 13:23:54.421299 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b36360-4443-4df8-b917-6c382b645f8e None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:54.421593 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b36360-4443-4df8-b917-6c382b645f8e None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:54.421855 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b36360-4443-4df8-b917-6c382b645f8e None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:54.423586 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 347/689] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:23:54.428616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce3364-038b-42c1-9e7c-9b653d19adc9 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:54.428942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce3364-038b-42c1-9e7c-9b653d19adc9 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:54.429226 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce3364-038b-42c1-9e7c-9b653d19adc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:54.447550 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-74ce3364-038b-42c1-9e7c-9b653d19adc9 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:54.448715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-74ce3364-038b-42c1-9e7c-9b653d19adc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:54.485355 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61106f-5ca5-45c7-8f97-129951da06a2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:54.494789 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:54.529189 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 348/690] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 13:23:54.530845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a8d57-5371-4c73-8b14-e68cefe9d0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:54.531449 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a8d57-5371-4c73-8b14-e68cefe9d0ea None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:54.531728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a8d57-5371-4c73-8b14-e68cefe9d0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:54.531998 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a8d57-5371-4c73-8b14-e68cefe9d0ea None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:54.532726 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a8d57-5371-4c73-8b14-e68cefe9d0ea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:54.533069 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a8d57-5371-4c73-8b14-e68cefe9d0ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:54.534600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a8d57-5371-4c73-8b14-e68cefe9d0ea None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:54.553566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 343/691] 10.222.0.23 () {56 vars in 1149 bytes} [Thu Oct 20 13:23:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:54.559301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce3364-038b-42c1-9e7c-9b653d19adc9 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:54.608800 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:54.669982 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce3364-038b-42c1-9e7c-9b653d19adc9 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:54.706844 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe527a-6286-4a3b-b3f9-ef11081ef230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:54.707425 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe527a-6286-4a3b-b3f9-ef11081ef230 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:54.707621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe527a-6286-4a3b-b3f9-ef11081ef230 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:54.707861 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe527a-6286-4a3b-b3f9-ef11081ef230 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:54.711494 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a8d57-5371-4c73-8b14-e68cefe9d0ea None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:54.786016 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe527a-6286-4a3b-b3f9-ef11081ef230 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:54.788146 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe527a-6286-4a3b-b3f9-ef11081ef230 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:54.837517 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:54.845606 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a8d57-5371-4c73-8b14-e68cefe9d0ea None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:54.879841 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850598a-4edd-4be1-ac9f-df12b414e274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:54.880464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850598a-4edd-4be1-ac9f-df12b414e274 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:54.880739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850598a-4edd-4be1-ac9f-df12b414e274 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:54.881044 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850598a-4edd-4be1-ac9f-df12b414e274 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:54.881622 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850598a-4edd-4be1-ac9f-df12b414e274 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:54.882981 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850598a-4edd-4be1-ac9f-df12b414e274 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:54.891758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-8850598a-4edd-4be1-ac9f-df12b414e274 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 13:23:54.891758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:54.891758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:23:54.891758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:23:54.891758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:23:54.891758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:23:54.891758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:23:54.891758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:54.891758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 13:23:54.891758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:54.891758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:23:54.891758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:54.891758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:54.891758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:54.891758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:54.891758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:54.891758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:54.892714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:54.892714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:54.892714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:54.892714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:54.892714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:54.892714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:54.892714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:23:54.892714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:23:54.892714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:23:54.892714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:23:54.892714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:54.892714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:54.892714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:54.892714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:54.892714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:54.892714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:54.894262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:54.894262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:54.894262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:54.894262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:54.894262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:54.894262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:54.894262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:54.894262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:54.894262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:54.894262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:54.894262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:23:54.894262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:54.894262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 13:23:54.894262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:54.894262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 344/692] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 20 13:23:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:23:54.895066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850598a-4edd-4be1-ac9f-df12b414e274 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:54.908158 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 349/693] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:23:54 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 13:23:54.908158 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe527a-6286-4a3b-b3f9-ef11081ef230 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:54.926442 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:55.002538 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850598a-4edd-4be1-ac9f-df12b414e274 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:55.010715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe527a-6286-4a3b-b3f9-ef11081ef230 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:55.039691 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78363350-968b-4043-8304-7d8dc889b268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:55.040319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78363350-968b-4043-8304-7d8dc889b268 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:55.040616 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78363350-968b-4043-8304-7d8dc889b268 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:55.040891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78363350-968b-4043-8304-7d8dc889b268 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:55.041465 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78363350-968b-4043-8304-7d8dc889b268 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:55.042823 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78363350-968b-4043-8304-7d8dc889b268 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:55.047104 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4b21a-ce06-4dde-82ea-eb4ea3f12d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:55.047701 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4b21a-ce06-4dde-82ea-eb4ea3f12d49 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:55.047977 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4b21a-ce06-4dde-82ea-eb4ea3f12d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:55.048246 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4b21a-ce06-4dde-82ea-eb4ea3f12d49 None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:55.048881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4b21a-ce06-4dde-82ea-eb4ea3f12d49 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:55.049193 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4b21a-ce06-4dde-82ea-eb4ea3f12d49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:55.051095 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4b21a-ce06-4dde-82ea-eb4ea3f12d49 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:55.052423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-78363350-968b-4043-8304-7d8dc889b268 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 13:23:55.052423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:55.052423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:23:55.052423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:23:55.052423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:23:55.052423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:23:55.052423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:23:55.052423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:55.052423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 13:23:55.052423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:55.052423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:23:55.052423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:55.052423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:55.052423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:55.052423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:55.052423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:55.052423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:55.053409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:55.053409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:55.053409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:55.053409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:55.053409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:55.053409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:55.053409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:23:55.053409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:23:55.053409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:23:55.053409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:23:55.053409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:55.053409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:55.053409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:55.053409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:55.053409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:55.053409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:55.054242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:55.054242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:55.054242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:55.054242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:55.054242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:55.054242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:55.054242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:55.054242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:55.054242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:55.054242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:55.054242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:23:55.054242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:55.054242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 13:23:55.054242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:55.054242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 345/694] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Oct 20 13:23:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:55.055878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78363350-968b-4043-8304-7d8dc889b268 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:55.061414 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 350/695] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 20 13:23:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:55.066631 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4b21a-ce06-4dde-82ea-eb4ea3f12d49 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:55.082946 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:55.152031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:55.161695 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78363350-968b-4043-8304-7d8dc889b268 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:55.181026 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4b21a-ce06-4dde-82ea-eb4ea3f12d49 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:55.197323 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8ef2a-5b18-4b5b-8a17-50f57ef79bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:55.197953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8ef2a-5b18-4b5b-8a17-50f57ef79bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:55.198232 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8ef2a-5b18-4b5b-8a17-50f57ef79bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:55.198504 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8ef2a-5b18-4b5b-8a17-50f57ef79bc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:55.210109 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a8ef2a-5b18-4b5b-8a17-50f57ef79bc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:55.211542 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a8ef2a-5b18-4b5b-8a17-50f57ef79bc5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:55.222120 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a5e88-f6d7-4e81-a33e-c2a8ad3c807a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:55.222919 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a5e88-f6d7-4e81-a33e-c2a8ad3c807a None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:55.223248 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a5e88-f6d7-4e81-a33e-c2a8ad3c807a None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:55.223523 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a5e88-f6d7-4e81-a33e-c2a8ad3c807a None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:55.224153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a5e88-f6d7-4e81-a33e-c2a8ad3c807a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:55.224453 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a5e88-f6d7-4e81-a33e-c2a8ad3c807a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:55.225829 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a5e88-f6d7-4e81-a33e-c2a8ad3c807a None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:55.226057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-67a8ef2a-5b18-4b5b-8a17-50f57ef79bc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:55.226057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:55.226057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:55.226057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:55.226057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:55.226057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:55.226057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:55.226057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:55.226057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:55.226057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:55.226057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:55.226057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:55.226057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:23:55.226057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:23:55.226057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:23:55.226057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:23:55.226951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:55.226951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:55.226951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:55.226951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:55.226951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:55.226951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:55.226951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:55.226951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:55.226951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:55.226951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:55.226951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:55.226951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:55.226951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:55.226951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:55.226951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:55.226951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:55.226951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:23:55.227856 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:23:55.227856 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:23:55.227856 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:55.227856 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 346/696] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:55.228750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8ef2a-5b18-4b5b-8a17-50f57ef79bc5 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:55.241164 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 351/697] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Oct 20 13:23:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:55.241164 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a5e88-f6d7-4e81-a33e-c2a8ad3c807a None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:55.320136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:55.340525 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a8ef2a-5b18-4b5b-8a17-50f57ef79bc5 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:55.347907 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a5e88-f6d7-4e81-a33e-c2a8ad3c807a None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:55.375707 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190de6a4-23ce-4672-9c18-4a51f3383cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:55.376313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190de6a4-23ce-4672-9c18-4a51f3383cba None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:55.376587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190de6a4-23ce-4672-9c18-4a51f3383cba None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:55.376906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190de6a4-23ce-4672-9c18-4a51f3383cba None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:55.377530 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190de6a4-23ce-4672-9c18-4a51f3383cba None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:55.377832 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190de6a4-23ce-4672-9c18-4a51f3383cba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:55.379256 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190de6a4-23ce-4672-9c18-4a51f3383cba None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:55.388923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 347/698] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 20 13:23:55 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:55.388923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190de6a4-23ce-4672-9c18-4a51f3383cba None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:55.390068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d081c-7628-4a3c-8cba-fb044a58b8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:55.390669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d081c-7628-4a3c-8cba-fb044a58b8cc None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:55.390951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d081c-7628-4a3c-8cba-fb044a58b8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:55.391221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d081c-7628-4a3c-8cba-fb044a58b8cc None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:55.391855 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779d081c-7628-4a3c-8cba-fb044a58b8cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:55.392154 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779d081c-7628-4a3c-8cba-fb044a58b8cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:55.393537 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779d081c-7628-4a3c-8cba-fb044a58b8cc None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:55.401997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:55.405943 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 352/699] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:55.411028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d081c-7628-4a3c-8cba-fb044a58b8cc None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:55.501262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190de6a4-23ce-4672-9c18-4a51f3383cba None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:55.523165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d081c-7628-4a3c-8cba-fb044a58b8cc None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:55.539227 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75880328-c96d-421d-b9a2-551ebeef6700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:55.539840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75880328-c96d-421d-b9a2-551ebeef6700 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:55.540122 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75880328-c96d-421d-b9a2-551ebeef6700 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:55.540400 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75880328-c96d-421d-b9a2-551ebeef6700 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:55.541066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75880328-c96d-421d-b9a2-551ebeef6700 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:55.541375 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75880328-c96d-421d-b9a2-551ebeef6700 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:55.542742 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75880328-c96d-421d-b9a2-551ebeef6700 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:55.553477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 348/700] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Oct 20 13:23:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:55.553477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75880328-c96d-421d-b9a2-551ebeef6700 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:55.569332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fca7e9-87b3-43b1-8402-1eac7b7a7186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:55.569940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fca7e9-87b3-43b1-8402-1eac7b7a7186 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:55.570217 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fca7e9-87b3-43b1-8402-1eac7b7a7186 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:55.570487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fca7e9-87b3-43b1-8402-1eac7b7a7186 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:55.571175 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fca7e9-87b3-43b1-8402-1eac7b7a7186 None admin] RBAC: Adding query filter params (interface=public, service_id=7bc8a15eea6e400db7b0f41f20329883, region_id=RegionOne) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:55.571483 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fca7e9-87b3-43b1-8402-1eac7b7a7186 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:55.572844 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fca7e9-87b3-43b1-8402-1eac7b7a7186 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:55.574389 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:55.578295 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 353/701] 10.222.0.23 () {56 vars in 1251 bytes} [Thu Oct 20 13:23:55 2022] GET /identity/v3/endpoints?service_id=7bc8a15eea6e400db7b0f41f20329883&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:55.579105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fca7e9-87b3-43b1-8402-1eac7b7a7186 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:55.638407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:55.672406 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75880328-c96d-421d-b9a2-551ebeef6700 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:55.684098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fca7e9-87b3-43b1-8402-1eac7b7a7186 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:55.710659 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6550d-f1d2-40b9-9115-6690e12bfb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:55.711283 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6550d-f1d2-40b9-9115-6690e12bfb93 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:55.711566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6550d-f1d2-40b9-9115-6690e12bfb93 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:55.711844 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6550d-f1d2-40b9-9115-6690e12bfb93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:55.717376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6c1d1-14c5-4c51-86e8-16088035a152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:55.717984 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6c1d1-14c5-4c51-86e8-16088035a152 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:55.718257 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6c1d1-14c5-4c51-86e8-16088035a152 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:55.718541 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6c1d1-14c5-4c51-86e8-16088035a152 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:55.719213 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6c1d1-14c5-4c51-86e8-16088035a152 None admin] RBAC: Adding query filter params (interface=public, service_id=8f212669ad614884b92f7e0ff110c3c9, region_id=RegionOne) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:55.719469 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f6550d-f1d2-40b9-9115-6690e12bfb93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:55.719584 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6c1d1-14c5-4c51-86e8-16088035a152 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:55.720895 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6c1d1-14c5-4c51-86e8-16088035a152 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:55.721437 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f6550d-f1d2-40b9-9115-6690e12bfb93 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:55.726364 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 354/702] 10.222.0.23 () {56 vars in 1251 bytes} [Thu Oct 20 13:23:55 2022] GET /identity/v3/endpoints?service_id=8f212669ad614884b92f7e0ff110c3c9&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:55.727145 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6c1d1-14c5-4c51-86e8-16088035a152 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:55.736774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-02f6550d-f1d2-40b9-9115-6690e12bfb93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:55.736774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:55.736774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:55.736774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:55.736774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:55.736774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:55.736774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:55.736774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:55.736774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:55.736774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:55.736774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:55.736774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:55.736774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:23:55.736774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:23:55.736774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:23:55.737934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:23:55.737934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:55.737934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:55.737934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:55.737934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:55.737934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:55.737934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:55.737934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:55.737934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:55.737934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:55.737934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:55.737934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:55.737934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:55.737934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:55.737934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:55.737934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:55.737934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:55.739028 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:23:55.739028 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:23:55.739028 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:23:55.739028 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:23:55.739028 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:23:55.739028 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:23:55.739028 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:23:55.739028 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:55.741008 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 349/703] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:55.741931 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6550d-f1d2-40b9-9115-6690e12bfb93 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:55.795198 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:55.833757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6c1d1-14c5-4c51-86e8-16088035a152 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:55.852079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f6550d-f1d2-40b9-9115-6690e12bfb93 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:55.868515 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8884dd1-cc1a-45bf-8857-104c912ccf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:55.869359 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8884dd1-cc1a-45bf-8857-104c912ccf2c None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:55.869787 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8884dd1-cc1a-45bf-8857-104c912ccf2c None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:55.870209 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8884dd1-cc1a-45bf-8857-104c912ccf2c None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:55.870982 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8884dd1-cc1a-45bf-8857-104c912ccf2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:55.872513 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8884dd1-cc1a-45bf-8857-104c912ccf2c None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:55.879679 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 355/704] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Oct 20 13:23:55 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:23:55.880636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8884dd1-cc1a-45bf-8857-104c912ccf2c None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:55.890023 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:55.897882 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb08fe-83df-466f-87d8-914a061e791d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:55.898877 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb08fe-83df-466f-87d8-914a061e791d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:55.899693 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb08fe-83df-466f-87d8-914a061e791d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:55.900086 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb08fe-83df-466f-87d8-914a061e791d None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:55.901471 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb08fe-83df-466f-87d8-914a061e791d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:55.903379 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb08fe-83df-466f-87d8-914a061e791d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:55.913048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 350/705] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Oct 20 13:23:55 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:23:56.013477 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8884dd1-cc1a-45bf-8857-104c912ccf2c None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:56.052773 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:56.062814 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564beb58-9c12-4ab8-aad2-f006379af640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:56.063587 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564beb58-9c12-4ab8-aad2-f006379af640 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:56.063875 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564beb58-9c12-4ab8-aad2-f006379af640 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:56.064158 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564beb58-9c12-4ab8-aad2-f006379af640 None admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:56.064957 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564beb58-9c12-4ab8-aad2-f006379af640 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:56.065282 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564beb58-9c12-4ab8-aad2-f006379af640 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:56.067007 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564beb58-9c12-4ab8-aad2-f006379af640 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:56.081736 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 356/706] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 20 13:23:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:56.088291 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb08fe-83df-466f-87d8-914a061e791d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:56.141288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3668b-a821-4367-97a0-2d2b863b8c1f None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:56.141592 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3668b-a821-4367-97a0-2d2b863b8c1f None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:56.141853 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3668b-a821-4367-97a0-2d2b863b8c1f None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:56.143478 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 357/707] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:56.149063 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8914127-a905-4279-833b-f04489c0657c None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:56.149351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8914127-a905-4279-833b-f04489c0657c None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:56.149648 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8914127-a905-4279-833b-f04489c0657c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:56.168771 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-b8914127-a905-4279-833b-f04489c0657c None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:56.169986 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b8914127-a905-4279-833b-f04489c0657c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:56.209854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb08fe-83df-466f-87d8-914a061e791d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:56.249016 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 358/708] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 13:23:56.250609 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfffc6e-d48b-4228-a40e-ab7ea1d5f6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:56.251213 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfffc6e-d48b-4228-a40e-ab7ea1d5f6bd None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:56.251489 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfffc6e-d48b-4228-a40e-ab7ea1d5f6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:56.251755 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfffc6e-d48b-4228-a40e-ab7ea1d5f6bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:56.252478 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfffc6e-d48b-4228-a40e-ab7ea1d5f6bd None admin] RBAC: Adding query filter params (role.id=ae10ab7a1f7f4eddae7ca60cc47d60d1, scope.project.id=4a81b6ecfb6e46a8b08c8bc2b4000fc7, user.id=85bb137d0fb74031a49a5c629b32c32b) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:56.252781 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfffc6e-d48b-4228-a40e-ab7ea1d5f6bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:56.255039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfffc6e-d48b-4228-a40e-ab7ea1d5f6bd None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:56.269784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 351/709] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:23:56 2022] GET /identity/v3/role_assignments?user.id=85bb137d0fb74031a49a5c629b32c32b&scope.project.id=4a81b6ecfb6e46a8b08c8bc2b4000fc7&role.id=ae10ab7a1f7f4eddae7ca60cc47d60d1 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:56.527046 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8914127-a905-4279-833b-f04489c0657c None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:56.632519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8914127-a905-4279-833b-f04489c0657c None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:56.650824 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:56.670344 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aacc17-0374-4f21-b1e5-1d7b8b192a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:56.671331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aacc17-0374-4f21-b1e5-1d7b8b192a1a None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:56.671840 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aacc17-0374-4f21-b1e5-1d7b8b192a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:56.672316 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aacc17-0374-4f21-b1e5-1d7b8b192a1a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:56.673137 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aacc17-0374-4f21-b1e5-1d7b8b192a1a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:56.674742 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aacc17-0374-4f21-b1e5-1d7b8b192a1a None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:56.684776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-60aacc17-0374-4f21-b1e5-1d7b8b192a1a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 13:23:56.684776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:56.684776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:23:56.684776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:23:56.684776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:23:56.684776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:23:56.684776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:23:56.684776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:56.684776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 13:23:56.684776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:56.684776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:23:56.684776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:56.684776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:56.684776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:56.684776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:56.684776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:56.684776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:56.685792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:56.685792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:56.685792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:56.685792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:56.685792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:56.685792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:56.685792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:23:56.685792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:23:56.685792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:23:56.685792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:23:56.685792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:56.685792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:56.685792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:56.685792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:56.685792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:56.685792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:56.686616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:56.686616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:56.686616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:56.686616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:56.686616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:56.686616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:56.686616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:56.686616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:56.686616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:56.686616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:56.686616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:23:56.686616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:56.686616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 13:23:56.686616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:56.689368 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 359/710] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:23:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:56.700912 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfffc6e-d48b-4228-a40e-ab7ea1d5f6bd None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:56.765820 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:56.811561 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfffc6e-d48b-4228-a40e-ab7ea1d5f6bd None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:56.850648 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50ee62-f465-4f3c-be77-d105ee0d58c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:56.851537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50ee62-f465-4f3c-be77-d105ee0d58c3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:56.852000 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50ee62-f465-4f3c-be77-d105ee0d58c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:56.852422 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50ee62-f465-4f3c-be77-d105ee0d58c3 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:56.853692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50ee62-f465-4f3c-be77-d105ee0d58c3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:56.854144 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50ee62-f465-4f3c-be77-d105ee0d58c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:56.855663 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f50ee62-f465-4f3c-be77-d105ee0d58c3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:56.861100 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 352/711] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 20 13:23:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:56.867688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aacc17-0374-4f21-b1e5-1d7b8b192a1a None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:56.975999 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aacc17-0374-4f21-b1e5-1d7b8b192a1a None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:57.011332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0416106-2044-43d7-8dab-7a66eaafc4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:57.012403 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0416106-2044-43d7-8dab-7a66eaafc4b4 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:57.012833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0416106-2044-43d7-8dab-7a66eaafc4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:57.013291 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0416106-2044-43d7-8dab-7a66eaafc4b4 None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:57.014091 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0416106-2044-43d7-8dab-7a66eaafc4b4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:57.014541 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0416106-2044-43d7-8dab-7a66eaafc4b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:57.016113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0416106-2044-43d7-8dab-7a66eaafc4b4 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:57.021533 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 360/712] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 20 13:23:56 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:58.245336 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b8e54-cf34-4ec7-83de-a8f5393fdcf1 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:58.245857 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b8e54-cf34-4ec7-83de-a8f5393fdcf1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:58.246280 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b8e54-cf34-4ec7-83de-a8f5393fdcf1 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:58.248954 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 353/713] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:58.254497 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5471304-eb5e-464f-a97b-4fad72f91d56 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:58.255304 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5471304-eb5e-464f-a97b-4fad72f91d56 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:58.255808 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5471304-eb5e-464f-a97b-4fad72f91d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:58.276297 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-c5471304-eb5e-464f-a97b-4fad72f91d56 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:58.277803 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-c5471304-eb5e-464f-a97b-4fad72f91d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:58.322697 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:58.354240 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 361/714] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:23:58.373887 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889af72d-fa03-4630-a0a6-2e58b4466e26 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:58.374387 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889af72d-fa03-4630-a0a6-2e58b4466e26 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:58.374861 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889af72d-fa03-4630-a0a6-2e58b4466e26 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:58.377565 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 354/715] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:58.386431 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b360c-3151-4e12-b701-5971d5ae207e None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:58.386881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b360c-3151-4e12-b701-5971d5ae207e None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:58.387610 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b360c-3151-4e12-b701-5971d5ae207e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:58.417297 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-f16b360c-3151-4e12-b701-5971d5ae207e None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:58.418860 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-f16b360c-3151-4e12-b701-5971d5ae207e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:58.520117 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effa5d9-61e4-4149-92eb-e577fcca97c7 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:58.520399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effa5d9-61e4-4149-92eb-e577fcca97c7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:58.520676 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effa5d9-61e4-4149-92eb-e577fcca97c7 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:58.523315 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 355/716] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:23:58.527406 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fc978-142d-4c9e-b1bb-d09c8016fc3e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:58.527689 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fc978-142d-4c9e-b1bb-d09c8016fc3e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:58.527959 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fc978-142d-4c9e-b1bb-d09c8016fc3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:58.537981 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 362/717] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:23:58.548572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c80fc978-142d-4c9e-b1bb-d09c8016fc3e None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:58.549866 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c80fc978-142d-4c9e-b1bb-d09c8016fc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:58.625780 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 356/718] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:23:58.652819 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b360c-3151-4e12-b701-5971d5ae207e None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:58.754397 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b360c-3151-4e12-b701-5971d5ae207e None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:58.756682 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:58.789785 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9246ac5-dae3-423a-bfb8-9e05a5e14712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:58.790338 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9246ac5-dae3-423a-bfb8-9e05a5e14712 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:58.790610 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9246ac5-dae3-423a-bfb8-9e05a5e14712 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:58.790875 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9246ac5-dae3-423a-bfb8-9e05a5e14712 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:58.791444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9246ac5-dae3-423a-bfb8-9e05a5e14712 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:58.792880 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9246ac5-dae3-423a-bfb8-9e05a5e14712 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:58.801636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-d9246ac5-dae3-423a-bfb8-9e05a5e14712 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 13:23:58.801636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:58.801636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:23:58.801636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:23:58.801636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:23:58.801636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:23:58.801636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:23:58.801636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:58.801636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 13:23:58.801636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:58.801636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:23:58.801636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:58.801636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:58.801636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:58.801636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:58.801636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:58.801636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:58.802583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:58.802583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:58.802583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:58.802583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:58.802583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:58.802583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:58.802583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:23:58.802583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:23:58.802583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:23:58.802583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:23:58.802583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:58.802583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:58.802583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:58.802583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:58.802583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:58.802583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:58.803421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:58.803421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:58.803421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:58.803421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:58.803421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:58.803421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:58.803421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:58.803421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:58.803421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:58.803421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:58.803421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:23:58.803421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:58.803421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 13:23:58.803421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:58.803421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 363/719] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Oct 20 13:23:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:58.810208 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80fc978-142d-4c9e-b1bb-d09c8016fc3e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:58.870352 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:58.881951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9246ac5-dae3-423a-bfb8-9e05a5e14712 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:58.922172 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80fc978-142d-4c9e-b1bb-d09c8016fc3e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:58.961932 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae694c7a-a934-472f-8beb-4e95d0698aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:58.963590 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae694c7a-a934-472f-8beb-4e95d0698aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:58.964073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae694c7a-a934-472f-8beb-4e95d0698aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:58.964519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae694c7a-a934-472f-8beb-4e95d0698aa9 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:58.965351 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae694c7a-a934-472f-8beb-4e95d0698aa9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:58.965826 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae694c7a-a934-472f-8beb-4e95d0698aa9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:58.967756 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae694c7a-a934-472f-8beb-4e95d0698aa9 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:58.974328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 357/720] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Oct 20 13:23:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:58.975405 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae694c7a-a934-472f-8beb-4e95d0698aa9 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:58.997685 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9246ac5-dae3-423a-bfb8-9e05a5e14712 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:59.035080 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226e3bc-acad-4dbe-8d9e-5dfaff9c9300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:59.035825 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226e3bc-acad-4dbe-8d9e-5dfaff9c9300 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:59.036131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226e3bc-acad-4dbe-8d9e-5dfaff9c9300 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:59.036488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226e3bc-acad-4dbe-8d9e-5dfaff9c9300 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:59.037094 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5226e3bc-acad-4dbe-8d9e-5dfaff9c9300 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:59.038575 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5226e3bc-acad-4dbe-8d9e-5dfaff9c9300 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:59.047889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-5226e3bc-acad-4dbe-8d9e-5dfaff9c9300 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 13:23:59.047889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:59.047889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:23:59.047889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:23:59.047889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:23:59.047889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:23:59.047889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:23:59.047889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:59.047889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 13:23:59.047889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:59.047889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:23:59.047889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:59.047889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:59.047889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:59.047889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:59.047889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:59.047889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:59.048848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:59.048848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:59.048848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:59.048848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:59.048848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:59.048848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:59.048848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:23:59.048848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:23:59.048848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:23:59.048848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:23:59.048848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:59.048848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:59.048848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:59.048848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:59.048848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:59.048848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:59.050281 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:59.050281 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:59.050281 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:59.050281 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:59.050281 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:59.050281 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:59.050281 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:59.050281 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:59.050281 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:59.050281 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:59.050281 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:23:59.050281 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:59.050281 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 20 13:23:59.050281 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:23:59.050281 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 364/721] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 20 13:23:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:23:59.051843 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226e3bc-acad-4dbe-8d9e-5dfaff9c9300 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:59.084409 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:59.087058 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae694c7a-a934-472f-8beb-4e95d0698aa9 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:59.106180 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:59.128178 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07e309-f07e-45ad-928b-39761dee4afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:59.128965 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07e309-f07e-45ad-928b-39761dee4afe None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:59.129613 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07e309-f07e-45ad-928b-39761dee4afe None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:59.129613 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07e309-f07e-45ad-928b-39761dee4afe None admin] PATH_INFO: `/v3/services/image` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:59.130192 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e07e309-f07e-45ad-928b-39761dee4afe None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:59.131688 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e07e309-f07e-45ad-928b-39761dee4afe None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:59.141319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-2e07e309-f07e-45ad-928b-39761dee4afe None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 13:23:59.141319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:59.141319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:23:59.141319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:23:59.141319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:23:59.141319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:23:59.141319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:23:59.141319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:59.141319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 13:23:59.141319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:59.141319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:23:59.141319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:59.141319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:23:59.141319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:23:59.141319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:23:59.141319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:23:59.141319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:23:59.142309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:23:59.142309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:23:59.142309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:23:59.142309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:23:59.142309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:23:59.142309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:23:59.142309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:23:59.142309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:23:59.142309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:23:59.142309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:23:59.142309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:23:59.142309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:23:59.142309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:23:59.142309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:23:59.142309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:23:59.142309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:23:59.146407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:23:59.146407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:23:59.146407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:23:59.146407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:23:59.146407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:23:59.146407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:23:59.146407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:23:59.146407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:23:59.146407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:23:59.146407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:23:59.146407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:23:59.146407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:23:59.146407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 13:23:59.146407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:23:59.146407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 358/722] 10.222.0.23 () {56 vars in 1105 bytes} [Thu Oct 20 13:23:58 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:23:59.147215 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07e309-f07e-45ad-928b-39761dee4afe None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:59.160805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226e3bc-acad-4dbe-8d9e-5dfaff9c9300 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:59.196754 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b27916f-1ba1-4e36-9d24-8a0be11518cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:59.197429 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27916f-1ba1-4e36-9d24-8a0be11518cf None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:59.197711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27916f-1ba1-4e36-9d24-8a0be11518cf None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:59.197986 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27916f-1ba1-4e36-9d24-8a0be11518cf None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:59.198624 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b27916f-1ba1-4e36-9d24-8a0be11518cf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:59.198930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b27916f-1ba1-4e36-9d24-8a0be11518cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:59.200305 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b27916f-1ba1-4e36-9d24-8a0be11518cf None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:59.205272 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 365/723] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Oct 20 13:23:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:59.205272 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b27916f-1ba1-4e36-9d24-8a0be11518cf None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:59.257229 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07e309-f07e-45ad-928b-39761dee4afe None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:59.293767 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a57060-264e-465e-a351-d1407ebcf7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:59.294379 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a57060-264e-465e-a351-d1407ebcf7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:59.294687 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a57060-264e-465e-a351-d1407ebcf7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:59.294939 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a57060-264e-465e-a351-d1407ebcf7e0 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:59.295570 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a57060-264e-465e-a351-d1407ebcf7e0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:59.296062 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a57060-264e-465e-a351-d1407ebcf7e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:59.297357 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a57060-264e-465e-a351-d1407ebcf7e0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:59.302848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:59.307361 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 359/724] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 20 13:23:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:59.308260 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a57060-264e-465e-a351-d1407ebcf7e0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:59.313279 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b27916f-1ba1-4e36-9d24-8a0be11518cf None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:59.351929 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529642d3-42f4-4247-b2d7-4eab8882649d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:59.352535 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529642d3-42f4-4247-b2d7-4eab8882649d None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:59.353023 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529642d3-42f4-4247-b2d7-4eab8882649d None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:59.353130 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529642d3-42f4-4247-b2d7-4eab8882649d None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:59.353740 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529642d3-42f4-4247-b2d7-4eab8882649d None admin] RBAC: Adding query filter params (name=image) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:59.354046 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529642d3-42f4-4247-b2d7-4eab8882649d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:59.355411 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529642d3-42f4-4247-b2d7-4eab8882649d None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:59.357402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:59.360897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 366/725] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:59.361642 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529642d3-42f4-4247-b2d7-4eab8882649d None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:59.424275 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a57060-264e-465e-a351-d1407ebcf7e0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:59.464267 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6b534-ba8b-4068-8f9e-25a61929e39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:59.464897 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6b534-ba8b-4068-8f9e-25a61929e39b None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:59.465186 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6b534-ba8b-4068-8f9e-25a61929e39b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:59.465465 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6b534-ba8b-4068-8f9e-25a61929e39b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:59.466137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6b534-ba8b-4068-8f9e-25a61929e39b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:59.467509 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6b534-ba8b-4068-8f9e-25a61929e39b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:59.471244 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529642d3-42f4-4247-b2d7-4eab8882649d None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:59.490918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 360/726] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:23:59 2022] POST /identity/v3/endpoints => generated 349 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:23:59.492113 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6b534-ba8b-4068-8f9e-25a61929e39b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:59.532318 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a29e6a-1467-4d6c-a486-12f9a4817a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:59.532964 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a29e6a-1467-4d6c-a486-12f9a4817a04 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:59.533244 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a29e6a-1467-4d6c-a486-12f9a4817a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:59.533522 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a29e6a-1467-4d6c-a486-12f9a4817a04 None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:59.534743 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a29e6a-1467-4d6c-a486-12f9a4817a04 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:59.535167 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a29e6a-1467-4d6c-a486-12f9a4817a04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:59.536589 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a29e6a-1467-4d6c-a486-12f9a4817a04 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:59.543807 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 367/727] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 20 13:23:59 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:59.545475 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bd8fd-6943-4376-bbfe-ea10b18bd681 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:59.545777 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bd8fd-6943-4376-bbfe-ea10b18bd681 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:59.546035 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bd8fd-6943-4376-bbfe-ea10b18bd681 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:59.547711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 368/728] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:23:59.552513 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bd8fd-6943-4376-bbfe-ea10b18bd681 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:23:59.575101 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:59.631533 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6b534-ba8b-4068-8f9e-25a61929e39b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:59.633369 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:23:59.666431 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bd8fd-6943-4376-bbfe-ea10b18bd681 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:23:59.671959 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc729f28-dc8a-4ddb-8bd7-d7443f5b5d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:59.672636 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc729f28-dc8a-4ddb-8bd7-d7443f5b5d17 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:59.673011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc729f28-dc8a-4ddb-8bd7-d7443f5b5d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:59.673292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc729f28-dc8a-4ddb-8bd7-d7443f5b5d17 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:59.673942 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc729f28-dc8a-4ddb-8bd7-d7443f5b5d17 None admin] RBAC: Adding query filter params (type=image) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:23:59.675404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc729f28-dc8a-4ddb-8bd7-d7443f5b5d17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:59.676804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc729f28-dc8a-4ddb-8bd7-d7443f5b5d17 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:59.683800 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 361/729] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:23:59 2022] GET /identity/v3/services?type=image => generated 122 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:23:59.692792 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427d954-38f2-4d83-b6b1-de08ebc4b1d7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:59.692792 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427d954-38f2-4d83-b6b1-de08ebc4b1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:59.692792 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7427d954-38f2-4d83-b6b1-de08ebc4b1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:59.707375 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7427d954-38f2-4d83-b6b1-de08ebc4b1d7 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:23:59.708611 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7427d954-38f2-4d83-b6b1-de08ebc4b1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:23:59.719877 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90395a48-8b8d-458a-8ad2-a8c13d54fb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:23:59.720719 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90395a48-8b8d-458a-8ad2-a8c13d54fb12 None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:23:59.720929 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90395a48-8b8d-458a-8ad2-a8c13d54fb12 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:23:59.721215 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90395a48-8b8d-458a-8ad2-a8c13d54fb12 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:23:59.721934 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90395a48-8b8d-458a-8ad2-a8c13d54fb12 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:23:59.723347 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90395a48-8b8d-458a-8ad2-a8c13d54fb12 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:23:59.756348 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 369/730] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:23:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:23:59.819679 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 362/731] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:23:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:24:00.089880 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90395a48-8b8d-458a-8ad2-a8c13d54fb12 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:00.171565 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:00.196972 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90395a48-8b8d-458a-8ad2-a8c13d54fb12 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:00.232471 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7657477d-3234-4ba5-989f-e804ee9e922f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:00.233218 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657477d-3234-4ba5-989f-e804ee9e922f None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:00.233564 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657477d-3234-4ba5-989f-e804ee9e922f None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:00.233906 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657477d-3234-4ba5-989f-e804ee9e922f None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:00.234636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7657477d-3234-4ba5-989f-e804ee9e922f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:00.236070 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7657477d-3234-4ba5-989f-e804ee9e922f None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:00.258304 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 370/732] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 20 13:24:00 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:24:01.732278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff89f63-f017-4261-8eed-6bb2ded21b66 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:01.733134 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff89f63-f017-4261-8eed-6bb2ded21b66 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:01.733579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff89f63-f017-4261-8eed-6bb2ded21b66 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:01.735297 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 363/733] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:24:01.741125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa9230-c704-483c-af3e-1354615a09c6 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:01.741941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa9230-c704-483c-af3e-1354615a09c6 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:01.742409 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa9230-c704-483c-af3e-1354615a09c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:01.761264 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-adfa9230-c704-483c-af3e-1354615a09c6 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:01.763017 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-adfa9230-c704-483c-af3e-1354615a09c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:01.846891 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 371/734] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:24:01.889703 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de750e-aaaf-4dc4-9615-d52e2e942b44 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:01.890337 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de750e-aaaf-4dc4-9615-d52e2e942b44 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:01.890808 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de750e-aaaf-4dc4-9615-d52e2e942b44 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:01.892606 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 364/735] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:24:01.898093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec86cd-c90b-44f9-885b-237afc74ee48 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:01.898573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec86cd-c90b-44f9-885b-237afc74ee48 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:01.899049 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec86cd-c90b-44f9-885b-237afc74ee48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:01.917665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-ccec86cd-c90b-44f9-885b-237afc74ee48 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:01.919226 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-ccec86cd-c90b-44f9-885b-237afc74ee48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:01.983365 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:01.996559 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 372/736] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:24:02.051852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a26610-0be8-4e0f-8373-7a05006460c3 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:02.052040 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a26610-0be8-4e0f-8373-7a05006460c3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:02.052327 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a26610-0be8-4e0f-8373-7a05006460c3 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:02.053964 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 365/737] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:24:02.059142 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c568f-6d6c-491f-aab5-6ee7445d623a None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:02.059440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c568f-6d6c-491f-aab5-6ee7445d623a None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:02.059717 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c568f-6d6c-491f-aab5-6ee7445d623a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:02.079247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-841c568f-6d6c-491f-aab5-6ee7445d623a None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:02.080441 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-841c568f-6d6c-491f-aab5-6ee7445d623a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:02.124269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a26610-0be8-4e0f-8373-7a05006460c3 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:02.152942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 373/738] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:24:02.226980 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:02.236596 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a26610-0be8-4e0f-8373-7a05006460c3 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:02.265296 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c568f-6d6c-491f-aab5-6ee7445d623a None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:02.279024 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2da1d6-e2ec-436b-b649-f4e73a54656f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:02.279930 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2da1d6-e2ec-436b-b649-f4e73a54656f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:02.280208 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2da1d6-e2ec-436b-b649-f4e73a54656f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:02.280956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2da1d6-e2ec-436b-b649-f4e73a54656f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:02.281563 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2da1d6-e2ec-436b-b649-f4e73a54656f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:02.283577 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2da1d6-e2ec-436b-b649-f4e73a54656f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:02.297018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-7d2da1d6-e2ec-436b-b649-f4e73a54656f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 13:24:02.297018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:02.297018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:24:02.297018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:24:02.297018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:24:02.297018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:24:02.297018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:24:02.297018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:24:02.297018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 13:24:02.297018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:24:02.297018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:24:02.297018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:24:02.297018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:02.297018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:02.297018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:02.297018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:02.297018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:02.298031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:02.298031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:02.298031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:02.298031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:02.298031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:02.298031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:02.298031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:24:02.298031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:24:02.298031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:24:02.298031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:24:02.298031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:02.298031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:02.298031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:02.298031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:02.298031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:02.298031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:02.298862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:02.298862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:02.298862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:02.298862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:02.298862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:02.298862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:02.298862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:02.298862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:02.298862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:02.298862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:02.298862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:24:02.298862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:24:02.298862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 13:24:02.298862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:24:02.300949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 366/739] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Oct 20 13:24:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:24:02.305972 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2da1d6-e2ec-436b-b649-f4e73a54656f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:02.372917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c568f-6d6c-491f-aab5-6ee7445d623a None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:02.398894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:02.413129 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a07374-f9a2-410c-9f83-e3f7e3ca8fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:02.414179 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a07374-f9a2-410c-9f83-e3f7e3ca8fd6 None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:02.414672 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a07374-f9a2-410c-9f83-e3f7e3ca8fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:02.415145 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a07374-f9a2-410c-9f83-e3f7e3ca8fd6 None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:02.416036 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a07374-f9a2-410c-9f83-e3f7e3ca8fd6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:02.417829 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a07374-f9a2-410c-9f83-e3f7e3ca8fd6 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:02.444959 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 374/740] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 20 13:24:02 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:24:02.444959 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a07374-f9a2-410c-9f83-e3f7e3ca8fd6 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:02.445250 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2da1d6-e2ec-436b-b649-f4e73a54656f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:02.496612 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763295a8-1ec5-415d-af3f-47ba71b0a66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:02.497254 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763295a8-1ec5-415d-af3f-47ba71b0a66a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:02.499817 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763295a8-1ec5-415d-af3f-47ba71b0a66a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:02.500128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763295a8-1ec5-415d-af3f-47ba71b0a66a None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:02.500789 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763295a8-1ec5-415d-af3f-47ba71b0a66a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:02.501124 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763295a8-1ec5-415d-af3f-47ba71b0a66a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:02.502526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763295a8-1ec5-415d-af3f-47ba71b0a66a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:02.511557 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 367/741] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Oct 20 13:24:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:24:02.518132 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763295a8-1ec5-415d-af3f-47ba71b0a66a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:02.541094 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:02.582063 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a07374-f9a2-410c-9f83-e3f7e3ca8fd6 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:02.619222 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0f2fa-fe85-408f-967f-c6ce42a630ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:02.619847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0f2fa-fe85-408f-967f-c6ce42a630ae None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:02.620127 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0f2fa-fe85-408f-967f-c6ce42a630ae None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:02.620398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0f2fa-fe85-408f-967f-c6ce42a630ae None admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:02.621095 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e0f2fa-fe85-408f-967f-c6ce42a630ae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:02.622551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e0f2fa-fe85-408f-967f-c6ce42a630ae None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:02.625208 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: WARNING keystone.server.flask.common [None req-98e0f2fa-fe85-408f-967f-c6ce42a630ae None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 20 13:24:02.642957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763295a8-1ec5-415d-af3f-47ba71b0a66a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:02.647555 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.sql.core [None req-98e0f2fa-fe85-408f-967f-c6ce42a630ae None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 20 13:24:02.647555 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 20 13:24:02.647555 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [parameters: {'id': '3740bfa3a1f1484caa0c1ad529839a77', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 20 13:24:02.647555 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81474) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 20 13:24:02.664646 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-98e0f2fa-fe85-408f-967f-c6ce42a630ae None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3740bfa3a1f1484caa0c1ad529839a77.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3740bfa3a1f1484caa0c1ad529839a77. Oct 20 13:24:02.664646 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:02.664646 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 13:24:02.664646 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 13:24:02.664646 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 13:24:02.664646 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 13:24:02.664646 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 13:24:02.664646 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 13:24:02.664646 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 13:24:02.664646 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application conn.query(q) Oct 20 13:24:02.664646 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 13:24:02.664646 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 13:24:02.664646 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 13:24:02.665748 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application result.read() Oct 20 13:24:02.665748 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 13:24:02.665748 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 13:24:02.665748 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 13:24:02.665748 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 13:24:02.665748 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 13:24:02.665748 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 13:24:02.665748 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 13:24:02.665748 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 13:24:02.665748 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 20 13:24:02.665748 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:02.665748 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 20 13:24:02.665748 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:02.665748 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:02.665748 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 20 13:24:02.665748 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 20 13:24:02.665748 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 20 13:24:02.665748 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 20 13:24:02.672122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 13:24:02.672122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application next(self.gen) Oct 20 13:24:02.672122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 20 13:24:02.672122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application yield resource Oct 20 13:24:02.672122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 13:24:02.672122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application next(self.gen) Oct 20 13:24:02.672122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 20 13:24:02.672122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self.session.rollback() Oct 20 13:24:02.672122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 20 13:24:02.672122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self.force_reraise() Oct 20 13:24:02.672122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 20 13:24:02.672122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise self.value Oct 20 13:24:02.672122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 20 13:24:02.672122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 20 13:24:02.672122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 20 13:24:02.672122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application session.commit() Oct 20 13:24:02.672122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 20 13:24:02.673288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 20 13:24:02.673288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 20 13:24:02.673288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._prepare_impl() Oct 20 13:24:02.673288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 20 13:24:02.673288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self.session.flush() Oct 20 13:24:02.673288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 20 13:24:02.673288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._flush(objects) Oct 20 13:24:02.673288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 20 13:24:02.673288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 20 13:24:02.673288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 20 13:24:02.673288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application compat.raise_( Oct 20 13:24:02.673288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 13:24:02.673288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception Oct 20 13:24:02.673288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 20 13:24:02.673288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application flush_context.execute() Oct 20 13:24:02.673288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 20 13:24:02.673288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rec.execute(self) Oct 20 13:24:02.674379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 20 13:24:02.674379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 20 13:24:02.674379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 20 13:24:02.674379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 20 13:24:02.674379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 20 13:24:02.674379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 20 13:24:02.674379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 20 13:24:02.674379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 20 13:24:02.674379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 20 13:24:02.674379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 20 13:24:02.674379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 20 13:24:02.674379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 20 13:24:02.674379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 20 13:24:02.674379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 20 13:24:02.674379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 20 13:24:02.676008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 20 13:24:02.676008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 13:24:02.676008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception Oct 20 13:24:02.676008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 13:24:02.676008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 13:24:02.676008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 13:24:02.676008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 13:24:02.676008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 13:24:02.676008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 13:24:02.676008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 13:24:02.676008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application conn.query(q) Oct 20 13:24:02.676008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 13:24:02.676008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 13:24:02.676008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 13:24:02.676008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application result.read() Oct 20 13:24:02.676008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 13:24:02.676008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 13:24:02.677174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 13:24:02.677174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 13:24:02.677174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 13:24:02.677174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 13:24:02.677174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 13:24:02.677174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 13:24:02.677174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 20 13:24:02.677174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 20 13:24:02.677174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [parameters: {'id': '3740bfa3a1f1484caa0c1ad529839a77', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 20 13:24:02.677174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 20 13:24:02.677174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:02.677174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:24:02.677174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:02.677174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:02.677174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 20 13:24:02.678165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 20 13:24:02.678165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 20 13:24:02.678165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 20 13:24:02.678165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 20 13:24:02.678165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:02.678165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:24:02.678165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:02.678165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:02.678165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:02.678165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:02.678165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:02.678165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:02.678165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:02.678165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:02.678165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:02.678165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:02.678165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:02.681111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:02.681111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 20 13:24:02.681111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 20 13:24:02.681111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:02.681111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:02.681111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 20 13:24:02.681111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 20 13:24:02.681111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3740bfa3a1f1484caa0c1ad529839a77. Oct 20 13:24:02.681111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:02.681111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 375/742] 10.222.0.23 () {60 vars in 1140 bytes} [Thu Oct 20 13:24:02 2022] POST /identity/v3/projects => generated 262 bytes in 225 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:24:02.681111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb2c93-ac6f-43e3-a133-a0119de07976 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:02.681111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb2c93-ac6f-43e3-a133-a0119de07976 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:02.682740 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb2c93-ac6f-43e3-a133-a0119de07976 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:02.682740 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 376/743] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:24:02.682740 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb2c93-ac6f-43e3-a133-a0119de07976 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:02.687752 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714a26d-14fe-4c03-bf97-acf67ececde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:02.688387 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714a26d-14fe-4c03-bf97-acf67ececde0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:02.688666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714a26d-14fe-4c03-bf97-acf67ececde0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:02.689023 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714a26d-14fe-4c03-bf97-acf67ececde0 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:02.689658 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f714a26d-14fe-4c03-bf97-acf67ececde0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:02.689998 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f714a26d-14fe-4c03-bf97-acf67ececde0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:02.691566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f714a26d-14fe-4c03-bf97-acf67ececde0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:02.696565 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 368/744] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Oct 20 13:24:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:02.697894 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2015a-9015-4336-9196-5788865286b7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:02.698179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2015a-9015-4336-9196-5788865286b7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:02.698448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2015a-9015-4336-9196-5788865286b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:02.717814 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f9c2015a-9015-4336-9196-5788865286b7 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:02.718988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f9c2015a-9015-4336-9196-5788865286b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:02.735861 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:02.793489 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 369/745] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:02 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:24:02.794343 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2015a-9015-4336-9196-5788865286b7 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:02.809231 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb2c93-ac6f-43e3-a133-a0119de07976 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:02.811060 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:02.850013 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb6e5a-288c-41c3-b5b1-e775f94dcfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:02.851127 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb6e5a-288c-41c3-b5b1-e775f94dcfc7 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:02.851606 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb6e5a-288c-41c3-b5b1-e775f94dcfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:02.852126 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb6e5a-288c-41c3-b5b1-e775f94dcfc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:02.859752 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb6e5a-288c-41c3-b5b1-e775f94dcfc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:02.861616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb6e5a-288c-41c3-b5b1-e775f94dcfc7 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:02.873973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-67fb6e5a-288c-41c3-b5b1-e775f94dcfc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:24:02.873973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:02.873973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:02.873973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:02.873973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:02.873973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:02.873973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:02.873973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:02.873973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:02.873973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:02.873973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:02.873973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:02.873973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:24:02.873973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:24:02.873973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:24:02.874851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:24:02.874851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:02.874851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:02.874851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:02.874851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:02.874851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:02.874851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:02.874851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:02.874851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:02.874851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:02.874851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:02.874851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:02.874851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:02.874851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:02.874851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:02.874851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:02.874851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:02.875711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:24:02.875711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:24:02.875711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:24:02.875711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:24:02.875711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:24:02.875711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:24:02.875711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:24:02.875711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:02.878206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 377/746] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:24:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:24:02.883896 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb6e5a-288c-41c3-b5b1-e775f94dcfc7 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:02.916379 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2015a-9015-4336-9196-5788865286b7 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:02.952367 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e66d13-dc9f-46c5-8a5e-a402a218b7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:02.953059 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e66d13-dc9f-46c5-8a5e-a402a218b7c2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:02.953447 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e66d13-dc9f-46c5-8a5e-a402a218b7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:02.953641 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e66d13-dc9f-46c5-8a5e-a402a218b7c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:02.954319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e66d13-dc9f-46c5-8a5e-a402a218b7c2 None admin] RBAC: Adding query filter params (interface=admin, service_id=8f212669ad614884b92f7e0ff110c3c9, region_id=RegionOne) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:02.954628 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e66d13-dc9f-46c5-8a5e-a402a218b7c2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:02.961194 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e66d13-dc9f-46c5-8a5e-a402a218b7c2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:02.967226 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 370/747] 10.222.0.23 () {56 vars in 1249 bytes} [Thu Oct 20 13:24:02 2022] GET /identity/v3/endpoints?service_id=8f212669ad614884b92f7e0ff110c3c9&interface=admin®ion_id=RegionOne => generated 193 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:02.968099 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e66d13-dc9f-46c5-8a5e-a402a218b7c2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:03.008406 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb6e5a-288c-41c3-b5b1-e775f94dcfc7 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:03.013199 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:03.054115 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b2107-500e-4010-a47e-24715701388d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:03.056174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b2107-500e-4010-a47e-24715701388d None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:03.056638 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b2107-500e-4010-a47e-24715701388d None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:03.057139 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b2107-500e-4010-a47e-24715701388d None admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:03.057979 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b2107-500e-4010-a47e-24715701388d None admin] RBAC: Adding query filter params (name=service) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:03.058457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b2107-500e-4010-a47e-24715701388d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:03.060052 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b2107-500e-4010-a47e-24715701388d None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:03.061196 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:03.080801 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 378/748] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 20 13:24:02 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:03.090485 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e66d13-dc9f-46c5-8a5e-a402a218b7c2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:03.100986 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b2107-500e-4010-a47e-24715701388d None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:03.136311 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744914de-a4ed-42c3-a517-e75917305810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:03.137282 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744914de-a4ed-42c3-a517-e75917305810 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:03.137642 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744914de-a4ed-42c3-a517-e75917305810 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:03.137953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744914de-a4ed-42c3-a517-e75917305810 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:03.138588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744914de-a4ed-42c3-a517-e75917305810 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:03.140022 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744914de-a4ed-42c3-a517-e75917305810 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:03.147935 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 371/749] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Oct 20 13:24:02 2022] GET /identity/v3/services => generated 1957 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:24:03.232643 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b2107-500e-4010-a47e-24715701388d None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:03.274330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da024978-d8b7-45d3-903b-af4ceb6a0faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:03.274979 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da024978-d8b7-45d3-903b-af4ceb6a0faf None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:03.275259 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da024978-d8b7-45d3-903b-af4ceb6a0faf None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:03.275529 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da024978-d8b7-45d3-903b-af4ceb6a0faf None admin] PATH_INFO: `/v3/services/compute` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:03.276118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da024978-d8b7-45d3-903b-af4ceb6a0faf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:03.277551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da024978-d8b7-45d3-903b-af4ceb6a0faf None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:03.287713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-da024978-d8b7-45d3-903b-af4ceb6a0faf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 13:24:03.287713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:03.287713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:24:03.287713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:24:03.287713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:24:03.287713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:24:03.287713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:24:03.287713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:24:03.287713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 13:24:03.287713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:03.287713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:24:03.287713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:03.287713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:03.287713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:03.287713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:03.287713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:03.287713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:03.288788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:03.288788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:03.288788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:03.288788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:03.288788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:03.288788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:03.288788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:24:03.288788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:24:03.288788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:24:03.288788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:24:03.288788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:03.288788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:03.288788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:03.288788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:03.288788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:03.288788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:03.289667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:03.289667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:03.289667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:03.289667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:03.289667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:03.289667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:03.289667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:03.289667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:03.289667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:03.289667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:03.289667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:24:03.289667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:24:03.289667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 13:24:03.289667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:03.289667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 379/750] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:24:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:24:03.295304 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744914de-a4ed-42c3-a517-e75917305810 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:03.416714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744914de-a4ed-42c3-a517-e75917305810 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:03.458496 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72a239-2ea6-4686-b63f-7ccf220f3ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:03.459113 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72a239-2ea6-4686-b63f-7ccf220f3ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:03.459399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72a239-2ea6-4686-b63f-7ccf220f3ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:03.459682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72a239-2ea6-4686-b63f-7ccf220f3ed1 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:03.460323 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72a239-2ea6-4686-b63f-7ccf220f3ed1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:03.460630 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72a239-2ea6-4686-b63f-7ccf220f3ed1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:03.462046 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72a239-2ea6-4686-b63f-7ccf220f3ed1 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:03.467610 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 372/751] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 20 13:24:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:03.473895 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da024978-d8b7-45d3-903b-af4ceb6a0faf None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:03.499117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:03.583621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da024978-d8b7-45d3-903b-af4ceb6a0faf None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:03.617479 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbb4cf-3c57-4dcf-b4e5-003f3dcd098c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:03.618085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbb4cf-3c57-4dcf-b4e5-003f3dcd098c None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:03.618368 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbb4cf-3c57-4dcf-b4e5-003f3dcd098c None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:03.618638 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbb4cf-3c57-4dcf-b4e5-003f3dcd098c None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:03.619260 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffbb4cf-3c57-4dcf-b4e5-003f3dcd098c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:03.619558 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffbb4cf-3c57-4dcf-b4e5-003f3dcd098c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:03.620941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffbb4cf-3c57-4dcf-b4e5-003f3dcd098c None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:03.626166 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 380/752] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 20 13:24:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:03.632144 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72a239-2ea6-4686-b63f-7ccf220f3ed1 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:03.635044 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:03.749283 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72a239-2ea6-4686-b63f-7ccf220f3ed1 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:03.787321 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0438254f-35e5-4891-9ee2-3313b6fc7880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:03.788372 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438254f-35e5-4891-9ee2-3313b6fc7880 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:03.788898 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438254f-35e5-4891-9ee2-3313b6fc7880 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:03.789372 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438254f-35e5-4891-9ee2-3313b6fc7880 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:03.790287 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0438254f-35e5-4891-9ee2-3313b6fc7880 None admin] RBAC: Adding query filter params (interface=public, service_id=c921e736149f4744981883a35657b4cd, region_id=RegionOne) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:03.790807 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0438254f-35e5-4891-9ee2-3313b6fc7880 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:03.792396 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0438254f-35e5-4891-9ee2-3313b6fc7880 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:03.798631 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 373/753] 10.222.0.23 () {56 vars in 1251 bytes} [Thu Oct 20 13:24:03 2022] GET /identity/v3/endpoints?service_id=c921e736149f4744981883a35657b4cd&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:03.805500 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbb4cf-3c57-4dcf-b4e5-003f3dcd098c None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:03.895096 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:03.914864 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbb4cf-3c57-4dcf-b4e5-003f3dcd098c None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:03.949840 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49755532-4f6c-43f7-84c3-8fe43e736c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:03.950446 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49755532-4f6c-43f7-84c3-8fe43e736c1d None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:03.950728 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49755532-4f6c-43f7-84c3-8fe43e736c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:03.951001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49755532-4f6c-43f7-84c3-8fe43e736c1d None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:03.951606 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49755532-4f6c-43f7-84c3-8fe43e736c1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:03.952995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49755532-4f6c-43f7-84c3-8fe43e736c1d None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:03.959938 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 381/754] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Oct 20 13:24:03 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:24:04.773028 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e836d-bcce-44fc-aa47-49cfef26f8ca None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:04.773379 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e836d-bcce-44fc-aa47-49cfef26f8ca None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:04.773503 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e836d-bcce-44fc-aa47-49cfef26f8ca None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:04.775108 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 374/755] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:24:04.780261 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f175b0-23f8-4fd9-98f1-53440e72cbce None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:04.780551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f175b0-23f8-4fd9-98f1-53440e72cbce None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:04.780828 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f175b0-23f8-4fd9-98f1-53440e72cbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:04.800102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-b7f175b0-23f8-4fd9-98f1-53440e72cbce None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:04.801631 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b7f175b0-23f8-4fd9-98f1-53440e72cbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:04.873740 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 382/756] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:04 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:24:05.148588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e836d-bcce-44fc-aa47-49cfef26f8ca None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:05.219015 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:05.260088 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e836d-bcce-44fc-aa47-49cfef26f8ca None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:05.298576 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb95a75-c318-4632-89ed-0cca2e76454e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:05.299206 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb95a75-c318-4632-89ed-0cca2e76454e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:05.299537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb95a75-c318-4632-89ed-0cca2e76454e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:05.299810 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb95a75-c318-4632-89ed-0cca2e76454e None admin] PATH_INFO: `/v3/services/image` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:05.300562 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb95a75-c318-4632-89ed-0cca2e76454e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:05.301955 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb95a75-c318-4632-89ed-0cca2e76454e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:05.311048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-4bb95a75-c318-4632-89ed-0cca2e76454e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 13:24:05.311048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:05.311048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:24:05.311048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:24:05.311048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:24:05.311048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:24:05.311048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:24:05.311048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:24:05.311048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 13:24:05.311048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:24:05.311048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:24:05.311048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:24:05.311048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:05.311048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:05.311048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:05.311048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:05.311048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:05.312050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:05.312050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:05.312050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:05.312050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:05.312050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:05.312050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:05.312050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:24:05.312050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:24:05.312050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:24:05.312050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:24:05.312050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:05.312050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:05.312050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:05.312050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:05.312050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:05.312050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:05.312924 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:05.312924 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:05.312924 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:05.312924 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:05.312924 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:05.312924 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:05.312924 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:05.312924 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:05.312924 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:05.312924 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:05.312924 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:24:05.312924 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:24:05.312924 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 13:24:05.312924 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:24:05.312924 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 375/757] 10.222.0.23 () {56 vars in 1105 bytes} [Thu Oct 20 13:24:05 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:24:05.319069 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f175b0-23f8-4fd9-98f1-53440e72cbce None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:05.360712 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c43de-a9f1-44c0-9b59-80bbd89750fd None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:05.361089 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c43de-a9f1-44c0-9b59-80bbd89750fd None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:05.361300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c43de-a9f1-44c0-9b59-80bbd89750fd None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:05.362745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 376/758] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:24:05.368479 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bee489-51f3-47f6-8904-23ef5af3d27f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:05.368803 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bee489-51f3-47f6-8904-23ef5af3d27f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:05.369060 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bee489-51f3-47f6-8904-23ef5af3d27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:05.389370 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-95bee489-51f3-47f6-8904-23ef5af3d27f None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:05.390573 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-95bee489-51f3-47f6-8904-23ef5af3d27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:05.413334 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:05.437462 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f175b0-23f8-4fd9-98f1-53440e72cbce None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:05.480945 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 377/759] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:05 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 13:24:05.482938 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b000ac0-a862-4223-826a-225e5df8685a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:05.483679 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b000ac0-a862-4223-826a-225e5df8685a None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:05.484078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b000ac0-a862-4223-826a-225e5df8685a None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:05.484397 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b000ac0-a862-4223-826a-225e5df8685a None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:05.485069 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b000ac0-a862-4223-826a-225e5df8685a None admin] RBAC: Adding query filter params (name=image) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:05.485374 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b000ac0-a862-4223-826a-225e5df8685a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:05.486800 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b000ac0-a862-4223-826a-225e5df8685a None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:05.490283 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61b56f-8b20-4f46-bee3-97ae1432b28d None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:05.490743 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61b56f-8b20-4f46-bee3-97ae1432b28d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:05.491165 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61b56f-8b20-4f46-bee3-97ae1432b28d None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:05.492896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 378/761] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:24:05.497949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f56b5c-8aa5-4cea-a975-d8a4829c9494 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:05.498422 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f56b5c-8aa5-4cea-a975-d8a4829c9494 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:05.498862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f56b5c-8aa5-4cea-a975-d8a4829c9494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:05.502179 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 383/761] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:24:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:05.505879 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b000ac0-a862-4223-826a-225e5df8685a None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:05.522937 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b0f56b5c-8aa5-4cea-a975-d8a4829c9494 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:05.524127 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b0f56b5c-8aa5-4cea-a975-d8a4829c9494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:05.598422 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:05.617222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 379/762] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:05 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 13:24:05.637638 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b000ac0-a862-4223-826a-225e5df8685a None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:05.673279 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc4433e-ae77-4bca-8665-60103daf8313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:05.674143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4433e-ae77-4bca-8665-60103daf8313 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:05.675165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4433e-ae77-4bca-8665-60103daf8313 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:05.675610 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4433e-ae77-4bca-8665-60103daf8313 None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:05.676466 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc4433e-ae77-4bca-8665-60103daf8313 None admin] RBAC: Adding query filter params (type=image) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:05.677049 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc4433e-ae77-4bca-8665-60103daf8313 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:05.678591 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc4433e-ae77-4bca-8665-60103daf8313 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:05.688789 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 384/763] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:24:05 2022] GET /identity/v3/services?type=image => generated 350 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:05.691378 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f56b5c-8aa5-4cea-a975-d8a4829c9494 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:05.741404 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc4433e-ae77-4bca-8665-60103daf8313 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:05.752645 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:05.811033 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f56b5c-8aa5-4cea-a975-d8a4829c9494 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:05.846276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc4433e-ae77-4bca-8665-60103daf8313 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:05.856647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c2b38-6316-48c7-9c85-f476d14da58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:05.857891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c2b38-6316-48c7-9c85-f476d14da58a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:05.858673 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c2b38-6316-48c7-9c85-f476d14da58a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:05.859142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c2b38-6316-48c7-9c85-f476d14da58a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:05.860278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c2b38-6316-48c7-9c85-f476d14da58a None admin] RBAC: Adding query filter params (interface=public, service_id=1d803358302c4195ac0de214fc380759, region_id=RegionOne) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:05.861119 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c2b38-6316-48c7-9c85-f476d14da58a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:05.863154 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c2b38-6316-48c7-9c85-f476d14da58a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:05.875371 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 380/764] 10.222.0.23 () {56 vars in 1251 bytes} [Thu Oct 20 13:24:05 2022] GET /identity/v3/endpoints?service_id=1d803358302c4195ac0de214fc380759&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:05.878021 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c2b38-6316-48c7-9c85-f476d14da58a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:05.893899 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfc55d-146d-4778-b96a-2a723542dbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:05.894547 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfc55d-146d-4778-b96a-2a723542dbbb None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:05.894807 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfc55d-146d-4778-b96a-2a723542dbbb None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:05.895080 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfc55d-146d-4778-b96a-2a723542dbbb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:05.895662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dfc55d-146d-4778-b96a-2a723542dbbb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:05.897145 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dfc55d-146d-4778-b96a-2a723542dbbb None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:05.905951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-36dfc55d-146d-4778-b96a-2a723542dbbb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 13:24:05.905951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:05.905951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:24:05.905951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:24:05.905951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:24:05.905951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:24:05.905951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:24:05.905951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:24:05.905951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 13:24:05.905951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:05.905951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:24:05.905951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:05.905951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:05.905951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:05.905951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:05.905951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:05.905951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:05.911916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:05.911916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:05.911916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:05.911916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:05.911916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:05.911916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:05.911916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:24:05.911916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:24:05.911916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:24:05.911916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:24:05.911916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:05.911916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:05.911916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:05.911916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:05.911916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:05.911916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:05.912994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:05.912994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:05.912994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:05.912994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:05.912994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:05.912994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:05.912994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:05.912994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:05.912994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:05.912994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:05.912994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:24:05.912994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:24:05.912994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 20 13:24:05.912994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:05.912994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 385/765] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Oct 20 13:24:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:24:05.913960 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfc55d-146d-4778-b96a-2a723542dbbb None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:05.946976 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:05.983193 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:05.995449 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c2b38-6316-48c7-9c85-f476d14da58a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:06.013910 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfc55d-146d-4778-b96a-2a723542dbbb None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:06.035712 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258869b-fc5d-4af1-bf28-62fd03c96167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:06.036325 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258869b-fc5d-4af1-bf28-62fd03c96167 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:06.036609 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258869b-fc5d-4af1-bf28-62fd03c96167 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:06.036906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258869b-fc5d-4af1-bf28-62fd03c96167 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:06.037527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258869b-fc5d-4af1-bf28-62fd03c96167 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:06.039011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258869b-fc5d-4af1-bf28-62fd03c96167 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:06.045827 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 381/766] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Oct 20 13:24:05 2022] GET /identity/v3/services => generated 1957 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:24:06.046632 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258869b-fc5d-4af1-bf28-62fd03c96167 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:06.055617 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64916636-943f-4f6f-93f7-b3fd04ce3ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:06.056670 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64916636-943f-4f6f-93f7-b3fd04ce3ed9 None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:06.057183 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64916636-943f-4f6f-93f7-b3fd04ce3ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:06.057717 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64916636-943f-4f6f-93f7-b3fd04ce3ed9 None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:06.058584 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64916636-943f-4f6f-93f7-b3fd04ce3ed9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:06.060445 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64916636-943f-4f6f-93f7-b3fd04ce3ed9 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:06.078423 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.sql.core [None req-64916636-943f-4f6f-93f7-b3fd04ce3ed9 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 20 13:24:06.078423 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 20 13:24:06.078423 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [parameters: {'user_id': 'e93d9dba3d9c4fd094cedecfa784a8a5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 20 13:24:06.078423 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81474) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 20 13:24:06.088252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-64916636-943f-4f6f-93f7-b3fd04ce3ed9 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 20 13:24:06.088252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:06.088252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 13:24:06.088252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 13:24:06.088252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 13:24:06.088252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 13:24:06.088252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 13:24:06.088252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 13:24:06.088252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 13:24:06.088252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application conn.query(q) Oct 20 13:24:06.088252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 13:24:06.088252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 13:24:06.088252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 13:24:06.088252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application result.read() Oct 20 13:24:06.088252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 13:24:06.089275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 13:24:06.089275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 13:24:06.089275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 13:24:06.089275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 13:24:06.089275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 13:24:06.089275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 13:24:06.089275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 13:24:06.089275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 20 13:24:06.089275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:06.089275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 20 13:24:06.089275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:06.089275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:06.089275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 20 13:24:06.089275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 20 13:24:06.089275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 20 13:24:06.089275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 20 13:24:06.089275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 13:24:06.089275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application next(self.gen) Oct 20 13:24:06.090195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 20 13:24:06.090195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application yield resource Oct 20 13:24:06.090195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 13:24:06.090195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application next(self.gen) Oct 20 13:24:06.090195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 20 13:24:06.090195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self.session.rollback() Oct 20 13:24:06.090195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 20 13:24:06.090195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self.force_reraise() Oct 20 13:24:06.090195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 20 13:24:06.090195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise self.value Oct 20 13:24:06.090195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 20 13:24:06.090195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 20 13:24:06.090195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 20 13:24:06.090195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application session.commit() Oct 20 13:24:06.090195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 20 13:24:06.090195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 20 13:24:06.090195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 20 13:24:06.091081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._prepare_impl() Oct 20 13:24:06.091081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 20 13:24:06.091081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self.session.flush() Oct 20 13:24:06.091081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 20 13:24:06.091081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._flush(objects) Oct 20 13:24:06.091081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 20 13:24:06.091081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 20 13:24:06.091081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 20 13:24:06.091081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application compat.raise_( Oct 20 13:24:06.091081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 13:24:06.091081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception Oct 20 13:24:06.091081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 20 13:24:06.091081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application flush_context.execute() Oct 20 13:24:06.091081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 20 13:24:06.091081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rec.execute(self) Oct 20 13:24:06.091081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 20 13:24:06.091081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 20 13:24:06.091948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 20 13:24:06.091948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 20 13:24:06.091948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 20 13:24:06.091948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 20 13:24:06.091948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 20 13:24:06.091948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 20 13:24:06.091948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 20 13:24:06.091948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 20 13:24:06.091948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 20 13:24:06.091948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 20 13:24:06.091948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 20 13:24:06.091948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 20 13:24:06.091948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 20 13:24:06.091948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 20 13:24:06.091948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 13:24:06.091948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception Oct 20 13:24:06.092805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 13:24:06.092805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 13:24:06.092805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 13:24:06.092805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 13:24:06.092805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 13:24:06.092805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 13:24:06.092805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 13:24:06.092805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application conn.query(q) Oct 20 13:24:06.092805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 13:24:06.092805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 13:24:06.092805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 13:24:06.092805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application result.read() Oct 20 13:24:06.092805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 13:24:06.092805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 13:24:06.092805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 13:24:06.092805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 13:24:06.093641 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 13:24:06.093641 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 13:24:06.093641 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 13:24:06.093641 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 13:24:06.093641 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 20 13:24:06.093641 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 20 13:24:06.093641 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e93d9dba3d9c4fd094cedecfa784a8a5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 20 13:24:06.093641 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 20 13:24:06.093641 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:06.093641 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:24:06.093641 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:06.093641 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:06.093641 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:06.093641 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:06.093641 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:06.094408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:06.094408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:06.094408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:06.094408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:06.094408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:06.094408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:06.094408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:06.094408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 20 13:24:06.094408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 20 13:24:06.094408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:06.094408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:06.094408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:24:06.094408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:24:06.094408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:24:06.094408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:24:06.094408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 20 13:24:06.094408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 20 13:24:06.095320 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 20 13:24:06.095320 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 20 13:24:06.095320 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 20 13:24:06.095320 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 20 13:24:06.095320 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 20 13:24:06.095320 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:06.095320 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 386/767] 10.222.0.23 () {60 vars in 1131 bytes} [Thu Oct 20 13:24:05 2022] POST /identity/v3/users => generated 160 bytes in 182 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:24:06.097318 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64916636-943f-4f6f-93f7-b3fd04ce3ed9 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:06.196966 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258869b-fc5d-4af1-bf28-62fd03c96167 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:06.216838 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:06.227997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64916636-943f-4f6f-93f7-b3fd04ce3ed9 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:06.238217 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff808be7-af4a-44d8-941e-5ddd949c9929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:06.238835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff808be7-af4a-44d8-941e-5ddd949c9929 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:06.239115 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff808be7-af4a-44d8-941e-5ddd949c9929 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:06.239388 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff808be7-af4a-44d8-941e-5ddd949c9929 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:06.240020 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff808be7-af4a-44d8-941e-5ddd949c9929 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:06.240319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff808be7-af4a-44d8-941e-5ddd949c9929 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:06.241714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff808be7-af4a-44d8-941e-5ddd949c9929 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:06.247371 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 382/768] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Oct 20 13:24:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:06.248627 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445939d-6a5e-4cef-8ae1-c5a8fa5152ab None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:06.248944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445939d-6a5e-4cef-8ae1-c5a8fa5152ab None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:06.249221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445939d-6a5e-4cef-8ae1-c5a8fa5152ab None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:06.250415 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 383/769] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:24:06.253702 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445939d-6a5e-4cef-8ae1-c5a8fa5152ab None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:06.267666 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b380284-6321-402c-b8e1-9416e8629a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:06.268637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b380284-6321-402c-b8e1-9416e8629a4b None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:06.268967 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b380284-6321-402c-b8e1-9416e8629a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:06.269420 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b380284-6321-402c-b8e1-9416e8629a4b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:06.271980 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:06.279828 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b380284-6321-402c-b8e1-9416e8629a4b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:06.281901 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b380284-6321-402c-b8e1-9416e8629a4b None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:06.293975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-0b380284-6321-402c-b8e1-9416e8629a4b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 20 13:24:06.293975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:06.293975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:06.293975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:06.293975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:06.293975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:06.293975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:06.293975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:06.293975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:06.293975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:06.293975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:06.293975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:06.293975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:24:06.293975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:24:06.293975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:24:06.293975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:24:06.294930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:06.294930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:06.294930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:24:06.294930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:24:06.294930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:24:06.294930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:24:06.294930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:06.294930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:06.294930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:06.294930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:06.294930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:06.294930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:06.294930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:06.294930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:06.294930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:06.294930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:06.294930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:06.294930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:06.296476 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:06.296476 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:06.296476 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:24:06.296476 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:24:06.296476 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:24:06.296476 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:24:06.296476 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:24:06.296476 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:24:06.296476 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 20 13:24:06.296476 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:06.296476 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 387/770] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:24:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:24:06.297113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a96956-e44f-47e8-b69f-44045edfdfcb None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:06.297113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a96956-e44f-47e8-b69f-44045edfdfcb None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:06.297591 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a96956-e44f-47e8-b69f-44045edfdfcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:06.315676 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-20a96956-e44f-47e8-b69f-44045edfdfcb None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:06.317260 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-20a96956-e44f-47e8-b69f-44045edfdfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:06.369531 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445939d-6a5e-4cef-8ae1-c5a8fa5152ab None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:06.391459 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 388/771] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:06 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:24:06.392190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a96956-e44f-47e8-b69f-44045edfdfcb None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:06.420844 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed932470-3271-4c47-abc6-175c1bddc660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:06.422473 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed932470-3271-4c47-abc6-175c1bddc660 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:06.423174 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed932470-3271-4c47-abc6-175c1bddc660 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:06.423862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed932470-3271-4c47-abc6-175c1bddc660 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:06.425618 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed932470-3271-4c47-abc6-175c1bddc660 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:06.426142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed932470-3271-4c47-abc6-175c1bddc660 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:06.428357 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed932470-3271-4c47-abc6-175c1bddc660 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:06.434315 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 384/772] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Oct 20 13:24:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:06.439617 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed932470-3271-4c47-abc6-175c1bddc660 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:06.505062 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:06.520527 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a96956-e44f-47e8-b69f-44045edfdfcb None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:06.546858 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:06.560683 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cae34a-90ec-4615-81ae-d5b381a950d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:06.561476 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cae34a-90ec-4615-81ae-d5b381a950d3 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:06.561906 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cae34a-90ec-4615-81ae-d5b381a950d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:06.562307 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cae34a-90ec-4615-81ae-d5b381a950d3 None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:06.563112 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cae34a-90ec-4615-81ae-d5b381a950d3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:06.563517 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cae34a-90ec-4615-81ae-d5b381a950d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:06.565123 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cae34a-90ec-4615-81ae-d5b381a950d3 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:06.577100 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed932470-3271-4c47-abc6-175c1bddc660 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:06.612891 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 389/773] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 20 13:24:06 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:06.617490 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff780327-947a-4f94-8a7d-14f0ab32685d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:06.618332 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff780327-947a-4f94-8a7d-14f0ab32685d None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:06.618712 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff780327-947a-4f94-8a7d-14f0ab32685d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:06.619139 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff780327-947a-4f94-8a7d-14f0ab32685d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:06.619982 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff780327-947a-4f94-8a7d-14f0ab32685d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:06.621531 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff780327-947a-4f94-8a7d-14f0ab32685d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:06.646170 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 385/774] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:24:06 2022] POST /identity/v3/endpoints => generated 325 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:24:06.667035 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cae34a-90ec-4615-81ae-d5b381a950d3 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:06.790800 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cae34a-90ec-4615-81ae-d5b381a950d3 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:06.840341 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31722b6-c284-4570-aedb-9f46de4e86ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:06.841032 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31722b6-c284-4570-aedb-9f46de4e86ca None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:06.841315 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31722b6-c284-4570-aedb-9f46de4e86ca None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:06.841590 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31722b6-c284-4570-aedb-9f46de4e86ca None admin] PATH_INFO: `/v3/services/compute` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:06.842185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31722b6-c284-4570-aedb-9f46de4e86ca None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:06.843578 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31722b6-c284-4570-aedb-9f46de4e86ca None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:06.853792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-d31722b6-c284-4570-aedb-9f46de4e86ca None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 13:24:06.853792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:06.853792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:24:06.853792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:24:06.853792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:24:06.853792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:24:06.853792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:24:06.853792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:24:06.853792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 13:24:06.853792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:06.853792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:24:06.853792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:06.853792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:06.853792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:06.853792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:06.853792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:06.853792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:06.854812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:06.854812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:06.854812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:06.854812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:06.854812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:06.854812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:06.854812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:24:06.854812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:24:06.854812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:24:06.854812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:24:06.854812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:06.854812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:06.854812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:06.854812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:06.854812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:06.854812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:06.856085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:06.856085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:06.856085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:06.856085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:06.856085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:06.856085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:06.856085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:06.856085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:06.856085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:06.856085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:06.856085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:24:06.856085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:24:06.856085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 20 13:24:06.856085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:06.856085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 390/775] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:24:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:24:06.861298 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff780327-947a-4f94-8a7d-14f0ab32685d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:06.981790 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff780327-947a-4f94-8a7d-14f0ab32685d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:06.989696 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:07.020551 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbcc2e-f33c-459c-a2e7-539d8e1cb290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:07.021326 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbcc2e-f33c-459c-a2e7-539d8e1cb290 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:07.021611 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbcc2e-f33c-459c-a2e7-539d8e1cb290 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:07.021884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbcc2e-f33c-459c-a2e7-539d8e1cb290 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:07.022516 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdbcc2e-f33c-459c-a2e7-539d8e1cb290 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:07.022816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdbcc2e-f33c-459c-a2e7-539d8e1cb290 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:07.024204 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdbcc2e-f33c-459c-a2e7-539d8e1cb290 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:07.029436 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 386/776] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 20 13:24:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:07.035735 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31722b6-c284-4570-aedb-9f46de4e86ca None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:07.079073 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:07.137621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31722b6-c284-4570-aedb-9f46de4e86ca None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:07.171113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f3aa6-18ec-4a2b-b9ad-6d62bdc858ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:07.171715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f3aa6-18ec-4a2b-b9ad-6d62bdc858ce None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:07.171991 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f3aa6-18ec-4a2b-b9ad-6d62bdc858ce None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:07.172261 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f3aa6-18ec-4a2b-b9ad-6d62bdc858ce None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:07.172898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067f3aa6-18ec-4a2b-b9ad-6d62bdc858ce None admin] RBAC: Adding query filter params (type=compute) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:07.173200 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067f3aa6-18ec-4a2b-b9ad-6d62bdc858ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:07.174571 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067f3aa6-18ec-4a2b-b9ad-6d62bdc858ce None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:07.179909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 391/777] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 20 13:24:07 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:07.185504 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbcc2e-f33c-459c-a2e7-539d8e1cb290 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:07.286190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbcc2e-f33c-459c-a2e7-539d8e1cb290 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:07.319845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ecab7-3c0f-4146-be52-f2193c93d45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:07.320447 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ecab7-3c0f-4146-be52-f2193c93d45c None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:07.320723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ecab7-3c0f-4146-be52-f2193c93d45c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:07.321022 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ecab7-3c0f-4146-be52-f2193c93d45c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:07.321687 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ecab7-3c0f-4146-be52-f2193c93d45c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:07.323054 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ecab7-3c0f-4146-be52-f2193c93d45c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:07.344480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 387/778] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:24:07 2022] POST /identity/v3/endpoints => generated 334 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:24:08.301066 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a0e0b-3105-447a-9fdb-0a79ac0b91a9 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:08.301608 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a0e0b-3105-447a-9fdb-0a79ac0b91a9 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:08.301608 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a0e0b-3105-447a-9fdb-0a79ac0b91a9 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:08.303278 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 392/779] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:24:08.308940 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155e924-b90a-4317-8bbb-8861eb4ffbbb None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:08.309443 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155e924-b90a-4317-8bbb-8861eb4ffbbb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:08.309514 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155e924-b90a-4317-8bbb-8861eb4ffbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:08.328930 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f155e924-b90a-4317-8bbb-8861eb4ffbbb None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:08.330125 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f155e924-b90a-4317-8bbb-8861eb4ffbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:08.352163 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:08.414169 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 388/780] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:24:08.683795 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a0e0b-3105-447a-9fdb-0a79ac0b91a9 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:08.789113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a0e0b-3105-447a-9fdb-0a79ac0b91a9 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:08.797219 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:08.828684 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b4482-3fc7-43a1-b52c-f1b173fe4cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:08.829346 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b4482-3fc7-43a1-b52c-f1b173fe4cdc None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:08.829630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b4482-3fc7-43a1-b52c-f1b173fe4cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:08.829903 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b4482-3fc7-43a1-b52c-f1b173fe4cdc None admin] PATH_INFO: `/v3/services/image` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:08.831023 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b4482-3fc7-43a1-b52c-f1b173fe4cdc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:08.832514 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b4482-3fc7-43a1-b52c-f1b173fe4cdc None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:08.841203 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-e07b4482-3fc7-43a1-b52c-f1b173fe4cdc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 13:24:08.841203 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:08.841203 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:24:08.841203 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:24:08.841203 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:24:08.841203 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:24:08.841203 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:24:08.841203 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:24:08.841203 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 13:24:08.841203 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:08.841203 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:24:08.841203 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:08.841203 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:08.841203 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:08.841203 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:08.841203 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:08.841203 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:08.842167 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:08.842167 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:08.842167 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:08.842167 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:08.842167 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:08.842167 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:08.842167 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:24:08.842167 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:24:08.842167 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:24:08.842167 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:24:08.842167 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:08.842167 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:08.842167 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:08.842167 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:08.842167 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:08.842167 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:08.843093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:08.843093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:08.843093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:08.843093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:08.843093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:08.843093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:08.843093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:08.843093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:08.843093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:08.843093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:08.843093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:24:08.843093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:24:08.843093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 20 13:24:08.843093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:08.845021 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 393/781] 10.222.0.23 () {56 vars in 1105 bytes} [Thu Oct 20 13:24:08 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:24:08.848966 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155e924-b90a-4317-8bbb-8861eb4ffbbb None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:08.898091 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc7881-1431-418e-8951-d283d1d6f9f0 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:08.898375 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc7881-1431-418e-8951-d283d1d6f9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:08.898657 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc7881-1431-418e-8951-d283d1d6f9f0 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:08.900126 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 394/782] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:24:08.904706 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdb5d6-581a-463b-bcc6-1e0c13c83278 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:08.905031 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdb5d6-581a-463b-bcc6-1e0c13c83278 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:08.905308 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdb5d6-581a-463b-bcc6-1e0c13c83278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:08.923658 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-c3cdb5d6-581a-463b-bcc6-1e0c13c83278 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:08.924924 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-c3cdb5d6-581a-463b-bcc6-1e0c13c83278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:08.952676 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155e924-b90a-4317-8bbb-8861eb4ffbbb None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:08.991814 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a3765-13a1-4cc3-be63-4cde1cefdcc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:08.992661 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a3765-13a1-4cc3-be63-4cde1cefdcc6 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:08.993153 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a3765-13a1-4cc3-be63-4cde1cefdcc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:08.993686 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a3765-13a1-4cc3-be63-4cde1cefdcc6 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:08.994467 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a3765-13a1-4cc3-be63-4cde1cefdcc6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:08.994706 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 395/783] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 13:24:08.995079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a3765-13a1-4cc3-be63-4cde1cefdcc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:08.996591 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a3765-13a1-4cc3-be63-4cde1cefdcc6 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:09.006949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 389/784] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:24:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:09.008293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdb5d6-581a-463b-bcc6-1e0c13c83278 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:09.107011 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:09.110072 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdb5d6-581a-463b-bcc6-1e0c13c83278 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:09.146546 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ca2b8-ab7e-4d99-a08e-cff13d3f733f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:09.147147 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ca2b8-ab7e-4d99-a08e-cff13d3f733f None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:09.147425 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ca2b8-ab7e-4d99-a08e-cff13d3f733f None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:09.147696 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ca2b8-ab7e-4d99-a08e-cff13d3f733f None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:09.148320 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ca2b8-ab7e-4d99-a08e-cff13d3f733f None admin] RBAC: Adding query filter params (type=image) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:09.148619 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ca2b8-ab7e-4d99-a08e-cff13d3f733f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:09.150045 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ca2b8-ab7e-4d99-a08e-cff13d3f733f None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:09.155074 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 396/785] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:24:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:24:09.160659 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a3765-13a1-4cc3-be63-4cde1cefdcc6 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:09.178958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:09.261123 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ca2b8-ab7e-4d99-a08e-cff13d3f733f None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:09.262526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a3765-13a1-4cc3-be63-4cde1cefdcc6 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:09.299843 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39847867-b460-4c25-bef3-137a0c6d78e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:09.300388 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39847867-b460-4c25-bef3-137a0c6d78e7 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:09.300677 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39847867-b460-4c25-bef3-137a0c6d78e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:09.300980 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39847867-b460-4c25-bef3-137a0c6d78e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:09.301658 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39847867-b460-4c25-bef3-137a0c6d78e7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:09.303022 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39847867-b460-4c25-bef3-137a0c6d78e7 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:09.332898 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 390/786] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:24:09 2022] POST /identity/v3/endpoints => generated 327 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:24:09.377215 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ca2b8-ab7e-4d99-a08e-cff13d3f733f None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:09.421917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d268d81-ebbb-48b0-9655-c1fcf5c70988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:09.422571 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d268d81-ebbb-48b0-9655-c1fcf5c70988 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:09.422845 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d268d81-ebbb-48b0-9655-c1fcf5c70988 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:09.423110 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d268d81-ebbb-48b0-9655-c1fcf5c70988 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:09.424803 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d268d81-ebbb-48b0-9655-c1fcf5c70988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:09.426699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d268d81-ebbb-48b0-9655-c1fcf5c70988 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:09.428387 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 397/787] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:24:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:24:09.434253 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39847867-b460-4c25-bef3-137a0c6d78e7 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:09.517843 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:09.555107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39847867-b460-4c25-bef3-137a0c6d78e7 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:09.592052 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319823a-2d4c-4c1f-ba23-d948708bc03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:09.592671 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319823a-2d4c-4c1f-ba23-d948708bc03b None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:09.592991 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319823a-2d4c-4c1f-ba23-d948708bc03b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:09.593265 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319823a-2d4c-4c1f-ba23-d948708bc03b None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:09.593950 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319823a-2d4c-4c1f-ba23-d948708bc03b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:09.595381 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319823a-2d4c-4c1f-ba23-d948708bc03b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:09.614936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9a8af-946f-48b9-aba6-b6ca68cc337a None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:09.615328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9a8af-946f-48b9-aba6-b6ca68cc337a None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:09.615641 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9a8af-946f-48b9-aba6-b6ca68cc337a None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:09.617324 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 398/788] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:24:09.622281 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201327f-af54-4688-85b8-019b180ef74d None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:09.622570 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201327f-af54-4688-85b8-019b180ef74d None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:09.622845 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201327f-af54-4688-85b8-019b180ef74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:09.625944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 391/789] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 20 13:24:09 2022] POST /identity/v3/projects => generated 311 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:24:09.642276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-a201327f-af54-4688-85b8-019b180ef74d None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:09.644225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-a201327f-af54-4688-85b8-019b180ef74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:09.671553 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:09.722539 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 399/790] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 13:24:09.991648 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319823a-2d4c-4c1f-ba23-d948708bc03b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:10.099025 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319823a-2d4c-4c1f-ba23-d948708bc03b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:10.123118 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:10.137452 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecdc88-7b7a-4b71-a950-75daef8556cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:10.138094 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecdc88-7b7a-4b71-a950-75daef8556cb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:10.138375 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecdc88-7b7a-4b71-a950-75daef8556cb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:10.138647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecdc88-7b7a-4b71-a950-75daef8556cb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:10.153494 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caecdc88-7b7a-4b71-a950-75daef8556cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:24:10.158487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-caecdc88-7b7a-4b71-a950-75daef8556cb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 20 13:24:10.158487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:10.158487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:10.158487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:10.158487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:10.158487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:10.158487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:10.158487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:10.158487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:10.158487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:10.158487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:10.158487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:10.158487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:24:10.158487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:24:10.158487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:24:10.158487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:24:10.159668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:24:10.159668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:24:10.159668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:24:10.159668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:24:10.159668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:10.159668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:10.159668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:10.159668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:10.159668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:10.159668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:10.159668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:10.159668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:10.159668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:10.159668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:10.159668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:10.159668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:10.160566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:10.160566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:10.160566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:10.160566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:10.160566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:24:10.160566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:24:10.160566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:24:10.160566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:24:10.160566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:24:10.160566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:24:10.160566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 20 13:24:10.160566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:24:10.160566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 392/791] 10.222.0.23 () {56 vars in 1120 bytes} [Thu Oct 20 13:24:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:24:10.165981 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201327f-af54-4688-85b8-019b180ef74d None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:10.266077 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201327f-af54-4688-85b8-019b180ef74d None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:10.300987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4ffb6-d037-4cc7-b41d-d3bc57cc8365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:10.301621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4ffb6-d037-4cc7-b41d-d3bc57cc8365 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:10.301903 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4ffb6-d037-4cc7-b41d-d3bc57cc8365 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:10.302180 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4ffb6-d037-4cc7-b41d-d3bc57cc8365 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:10.302872 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4ffb6-d037-4cc7-b41d-d3bc57cc8365 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:10.303180 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4ffb6-d037-4cc7-b41d-d3bc57cc8365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:10.304608 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4ffb6-d037-4cc7-b41d-d3bc57cc8365 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:10.318709 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 400/792] 10.222.0.23 () {56 vars in 1115 bytes} [Thu Oct 20 13:24:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:24:10.320899 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecdc88-7b7a-4b71-a950-75daef8556cb None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:10.425690 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecdc88-7b7a-4b71-a950-75daef8556cb None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:10.462487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d380b5-667b-450d-847c-b237f9f25ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:10.463351 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d380b5-667b-450d-847c-b237f9f25ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:10.463798 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d380b5-667b-450d-847c-b237f9f25ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:10.464223 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d380b5-667b-450d-847c-b237f9f25ef3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:10.475265 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d380b5-667b-450d-847c-b237f9f25ef3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:10.476905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d380b5-667b-450d-847c-b237f9f25ef3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:10.491949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-79d380b5-667b-450d-847c-b237f9f25ef3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:24:10.491949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:10.491949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:10.491949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:10.491949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:10.491949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:10.491949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:10.491949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:10.491949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:10.491949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:10.491949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:10.491949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:10.491949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:24:10.491949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:24:10.491949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:24:10.491949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:24:10.493120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:10.493120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:10.493120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:10.493120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:10.493120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:10.493120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:10.493120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:10.493120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:10.493120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:10.493120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:10.493120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:10.493120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:10.493120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:10.493120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:10.493120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:10.493120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:10.493120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:24:10.494220 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:24:10.494220 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:24:10.494220 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:24:10.495871 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 393/793] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:24:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:24:10.501311 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4ffb6-d037-4cc7-b41d-d3bc57cc8365 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:10.526629 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:10.610335 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4ffb6-d037-4cc7-b41d-d3bc57cc8365 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:10.645037 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76362219-ce34-47f2-acd2-df03338bfada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:10.645651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76362219-ce34-47f2-acd2-df03338bfada None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:10.645934 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76362219-ce34-47f2-acd2-df03338bfada None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:10.646210 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76362219-ce34-47f2-acd2-df03338bfada None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:10.646851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76362219-ce34-47f2-acd2-df03338bfada None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:10.647156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76362219-ce34-47f2-acd2-df03338bfada None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:10.648530 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76362219-ce34-47f2-acd2-df03338bfada None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:10.665433 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 401/794] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:24:10.666564 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d380b5-667b-450d-847c-b237f9f25ef3 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:10.703248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:10.769799 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d380b5-667b-450d-847c-b237f9f25ef3 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:10.804178 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6acf2-818f-408f-8bed-2e3867cd4c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:10.804798 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6acf2-818f-408f-8bed-2e3867cd4c2d None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:10.805103 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6acf2-818f-408f-8bed-2e3867cd4c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:10.805371 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6acf2-818f-408f-8bed-2e3867cd4c2d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:10.812694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6acf2-818f-408f-8bed-2e3867cd4c2d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:10.814145 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6acf2-818f-408f-8bed-2e3867cd4c2d None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:10.825425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-91b6acf2-818f-408f-8bed-2e3867cd4c2d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 20 13:24:10.825425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:10.825425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:10.825425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:10.825425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:10.825425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:10.825425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:10.825425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:10.825425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:10.825425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:10.825425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:10.825425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:10.825425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:24:10.825425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:24:10.825425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:24:10.825425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:24:10.826383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:10.826383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:10.826383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:24:10.826383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:24:10.826383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:24:10.826383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:24:10.826383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:10.826383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:10.826383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:10.826383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:10.826383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:10.826383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:10.826383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:10.826383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:10.826383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:10.826383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:10.826383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:10.826383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:10.827343 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:10.827343 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:10.827343 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:24:10.827343 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:24:10.827343 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:24:10.827343 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:24:10.827343 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:24:10.827343 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:24:10.827343 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 20 13:24:10.827343 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:24:10.829196 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 394/795] 10.222.0.23 () {56 vars in 1093 bytes} [Thu Oct 20 13:24:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:24:10.832929 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76362219-ce34-47f2-acd2-df03338bfada None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:10.915692 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:10.936842 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76362219-ce34-47f2-acd2-df03338bfada None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:10.970501 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac98a5-0cf9-4426-be63-8d2f2c672712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:10.971128 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac98a5-0cf9-4426-be63-8d2f2c672712 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:10.971397 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac98a5-0cf9-4426-be63-8d2f2c672712 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:10.971663 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac98a5-0cf9-4426-be63-8d2f2c672712 None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:10.972374 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac98a5-0cf9-4426-be63-8d2f2c672712 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:10.972671 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac98a5-0cf9-4426-be63-8d2f2c672712 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:10.974124 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac98a5-0cf9-4426-be63-8d2f2c672712 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:10.995865 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 402/796] 10.222.0.23 () {56 vars in 1133 bytes} [Thu Oct 20 13:24:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:10.997380 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6acf2-818f-408f-8bed-2e3867cd4c2d None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:11.095415 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:11.102409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6acf2-818f-408f-8bed-2e3867cd4c2d None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:11.136805 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c0f96-188b-4b1d-bc2e-a57b0dc26c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:11.137717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c0f96-188b-4b1d-bc2e-a57b0dc26c14 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:11.137999 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c0f96-188b-4b1d-bc2e-a57b0dc26c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:11.138273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c0f96-188b-4b1d-bc2e-a57b0dc26c14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:11.149172 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c0f96-188b-4b1d-bc2e-a57b0dc26c14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:11.150671 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c0f96-188b-4b1d-bc2e-a57b0dc26c14 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:11.164923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d95c0f96-188b-4b1d-bc2e-a57b0dc26c14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:24:11.164923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:11.164923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:11.164923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:11.164923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:11.164923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:11.164923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:11.164923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:11.164923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:11.164923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:11.164923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:11.164923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:11.164923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:24:11.164923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:24:11.164923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:24:11.164923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:24:11.165852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:11.165852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:11.165852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:11.165852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:11.165852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:11.165852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:11.165852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:11.165852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:11.165852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:11.165852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:11.165852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:11.165852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:11.165852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:11.165852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:11.165852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:11.165852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:11.165852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:24:11.166767 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:24:11.166767 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:24:11.166767 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:24:11.170828 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 395/797] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:24:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:24:11.172038 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac98a5-0cf9-4426-be63-8d2f2c672712 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:11.274797 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac98a5-0cf9-4426-be63-8d2f2c672712 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:11.308326 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad2f5e-7fdd-4dc3-913a-c76f1483e3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:11.308993 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad2f5e-7fdd-4dc3-913a-c76f1483e3f9 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:11.309278 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad2f5e-7fdd-4dc3-913a-c76f1483e3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:11.309553 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad2f5e-7fdd-4dc3-913a-c76f1483e3f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:11.310196 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad2f5e-7fdd-4dc3-913a-c76f1483e3f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:11.310500 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad2f5e-7fdd-4dc3-913a-c76f1483e3f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:11.311870 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad2f5e-7fdd-4dc3-913a-c76f1483e3f9 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:11.318829 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:11.329840 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 403/798] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:11.332243 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c0f96-188b-4b1d-bc2e-a57b0dc26c14 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:11.435515 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c0f96-188b-4b1d-bc2e-a57b0dc26c14 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:11.469787 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc9084-9631-458d-9139-0ff840d87d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:11.470425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc9084-9631-458d-9139-0ff840d87d63 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:11.470728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc9084-9631-458d-9139-0ff840d87d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:11.471584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc9084-9631-458d-9139-0ff840d87d63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:11.478566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc9084-9631-458d-9139-0ff840d87d63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:11.480018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc9084-9631-458d-9139-0ff840d87d63 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:11.492215 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-a1cc9084-9631-458d-9139-0ff840d87d63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:24:11.492215 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:11.492215 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:11.492215 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:11.492215 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:11.492215 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:11.492215 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:11.492215 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:11.492215 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:11.492215 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:11.492215 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:11.492215 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:11.492215 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:24:11.492215 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:24:11.492215 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:24:11.493221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:24:11.493221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:11.493221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:11.493221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:11.493221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:11.493221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:11.493221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:11.493221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:11.493221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:11.493221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:11.493221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:11.493221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:11.493221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:11.493221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:11.493221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:11.493221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:11.493221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:11.494091 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:24:11.494091 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:24:11.494091 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:24:11.494091 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:24:11.494091 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:24:11.494091 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:24:11.494091 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:24:11.494091 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:24:11.498730 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 396/799] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:24:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:24:11.499794 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad2f5e-7fdd-4dc3-913a-c76f1483e3f9 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:11.609694 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad2f5e-7fdd-4dc3-913a-c76f1483e3f9 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:11.613318 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ba110-7e40-483f-a8fb-9958ff29882a None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:11.613721 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ba110-7e40-483f-a8fb-9958ff29882a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:11.613999 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ba110-7e40-483f-a8fb-9958ff29882a None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:11.615470 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 397/800] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:24:11.620982 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a2e69-d712-42cf-a7eb-3322dd51fd3b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:11.621636 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a2e69-d712-42cf-a7eb-3322dd51fd3b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:11.622213 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a2e69-d712-42cf-a7eb-3322dd51fd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:11.641090 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-819a2e69-d712-42cf-a7eb-3322dd51fd3b None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:11.642269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-819a2e69-d712-42cf-a7eb-3322dd51fd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:11.645082 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:11.650447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab11b6-fc9e-4f37-a3bc-8187009b986d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:11.651055 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab11b6-fc9e-4f37-a3bc-8187009b986d None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:11.651380 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab11b6-fc9e-4f37-a3bc-8187009b986d None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:11.651653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab11b6-fc9e-4f37-a3bc-8187009b986d None admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:11.652338 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab11b6-fc9e-4f37-a3bc-8187009b986d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:11.652642 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab11b6-fc9e-4f37-a3bc-8187009b986d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:11.654108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab11b6-fc9e-4f37-a3bc-8187009b986d None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:11.666764 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 404/801] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 20 13:24:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:11.672521 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab11b6-fc9e-4f37-a3bc-8187009b986d None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:11.720238 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 398/802] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 13:24:11.740335 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:11.785198 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab11b6-fc9e-4f37-a3bc-8187009b986d None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:11.819590 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c927a1c0-dd43-47ab-aa23-8701cd28ad67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:11.820245 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927a1c0-dd43-47ab-aa23-8701cd28ad67 None admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:11.820522 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927a1c0-dd43-47ab-aa23-8701cd28ad67 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:11.820792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927a1c0-dd43-47ab-aa23-8701cd28ad67 None admin] PATH_INFO: `/v3/projects/4a81b6ecfb6e46a8b08c8bc2b4000fc7/users/ee5c2a3855cb4a668177aa19f337778d/roles/ae10ab7a1f7f4eddae7ca60cc47d60d1` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:11.823122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c927a1c0-dd43-47ab-aa23-8701cd28ad67 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a81b6ecfb6e46a8b08c8bc2b4000fc7, user_id=ee5c2a3855cb4a668177aa19f337778d, role_id=ae10ab7a1f7f4eddae7ca60cc47d60d1)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:11.824608 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c927a1c0-dd43-47ab-aa23-8701cd28ad67 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:11.831414 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49954aac-e21e-41e7-912d-ef4eaa035684 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:11.831747 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49954aac-e21e-41e7-912d-ef4eaa035684 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:11.832164 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49954aac-e21e-41e7-912d-ef4eaa035684 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:11.833748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 399/803] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:24:11.838838 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c689d5-37f3-4657-8ac7-ffea482777a7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:11.839146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c689d5-37f3-4657-8ac7-ffea482777a7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:11.839414 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c689d5-37f3-4657-8ac7-ffea482777a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:11.839830 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 405/804] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:24:11 2022] PUT /identity/v3/projects/4a81b6ecfb6e46a8b08c8bc2b4000fc7/users/ee5c2a3855cb4a668177aa19f337778d/roles/ae10ab7a1f7f4eddae7ca60cc47d60d1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:24:11.859423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-98c689d5-37f3-4657-8ac7-ffea482777a7 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:11.860678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-98c689d5-37f3-4657-8ac7-ffea482777a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:11.938767 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 400/805] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:24:11.996278 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c927a1c0-dd43-47ab-aa23-8701cd28ad67 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:12.100151 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c927a1c0-dd43-47ab-aa23-8701cd28ad67 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:12.118174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:12.136372 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb0b70-80a2-4eb2-9f88-cadf935e7fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:12.137391 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb0b70-80a2-4eb2-9f88-cadf935e7fac None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:12.137742 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb0b70-80a2-4eb2-9f88-cadf935e7fac None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:12.138082 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb0b70-80a2-4eb2-9f88-cadf935e7fac None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:12.138807 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb0b70-80a2-4eb2-9f88-cadf935e7fac None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:12.140242 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb0b70-80a2-4eb2-9f88-cadf935e7fac None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:12.150036 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.sql.core [None req-71cb0b70-80a2-4eb2-9f88-cadf935e7fac None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 20 13:24:12.150036 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 20 13:24:12.150036 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [parameters: {'id': 'a234cb9fe7654d238ca8458fc1c6c802', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 20 13:24:12.150036 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81474) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 20 13:24:12.159002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-71cb0b70-80a2-4eb2-9f88-cadf935e7fac None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 20 13:24:12.159002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:12.159002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 13:24:12.159002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 13:24:12.159002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 13:24:12.159002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 13:24:12.159002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 13:24:12.159002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 13:24:12.159002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 13:24:12.159002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application conn.query(q) Oct 20 13:24:12.159002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 13:24:12.159002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 13:24:12.159002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 13:24:12.159002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application result.read() Oct 20 13:24:12.160157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 13:24:12.160157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 13:24:12.160157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 13:24:12.160157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 13:24:12.160157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 13:24:12.160157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 13:24:12.160157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 13:24:12.160157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 13:24:12.160157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 20 13:24:12.160157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:12.160157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 20 13:24:12.160157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:12.160157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:12.160157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 20 13:24:12.160157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 20 13:24:12.160157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 20 13:24:12.160157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 20 13:24:12.160157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 13:24:12.161439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application next(self.gen) Oct 20 13:24:12.161439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 20 13:24:12.161439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application yield resource Oct 20 13:24:12.161439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 20 13:24:12.161439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application next(self.gen) Oct 20 13:24:12.161439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 20 13:24:12.161439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self.session.rollback() Oct 20 13:24:12.161439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 20 13:24:12.161439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self.force_reraise() Oct 20 13:24:12.161439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 20 13:24:12.161439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise self.value Oct 20 13:24:12.161439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 20 13:24:12.161439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 20 13:24:12.161439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 20 13:24:12.161439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application session.commit() Oct 20 13:24:12.161439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 20 13:24:12.161439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 20 13:24:12.162548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 20 13:24:12.162548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._prepare_impl() Oct 20 13:24:12.162548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 20 13:24:12.162548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self.session.flush() Oct 20 13:24:12.162548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 20 13:24:12.162548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._flush(objects) Oct 20 13:24:12.162548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 20 13:24:12.162548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 20 13:24:12.162548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 20 13:24:12.162548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application compat.raise_( Oct 20 13:24:12.162548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 13:24:12.162548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception Oct 20 13:24:12.162548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 20 13:24:12.162548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application flush_context.execute() Oct 20 13:24:12.162548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 20 13:24:12.162548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rec.execute(self) Oct 20 13:24:12.162548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 20 13:24:12.165083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 20 13:24:12.165083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 20 13:24:12.165083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 20 13:24:12.165083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 20 13:24:12.165083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 20 13:24:12.165083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 20 13:24:12.165083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 20 13:24:12.165083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 20 13:24:12.165083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 20 13:24:12.165083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 20 13:24:12.165083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 20 13:24:12.165083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 20 13:24:12.165083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 20 13:24:12.165083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 20 13:24:12.165083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 20 13:24:12.165936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 20 13:24:12.165936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception Oct 20 13:24:12.165936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 20 13:24:12.165936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 20 13:24:12.165936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 20 13:24:12.165936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 20 13:24:12.165936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 20 13:24:12.165936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application result = self._query(query) Oct 20 13:24:12.165936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 20 13:24:12.165936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application conn.query(q) Oct 20 13:24:12.165936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 20 13:24:12.165936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 20 13:24:12.165936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 20 13:24:12.165936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application result.read() Oct 20 13:24:12.165936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 20 13:24:12.165936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 20 13:24:12.165936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 20 13:24:12.166821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 20 13:24:12.166821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 20 13:24:12.166821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 20 13:24:12.166821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 20 13:24:12.166821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 20 13:24:12.166821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 20 13:24:12.166821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 20 13:24:12.166821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [parameters: {'id': 'a234cb9fe7654d238ca8458fc1c6c802', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 20 13:24:12.166821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 20 13:24:12.166821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:12.166821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:24:12.166821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:12.166821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:12.166821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 20 13:24:12.166821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 20 13:24:12.167659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 20 13:24:12.167659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 20 13:24:12.167659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 20 13:24:12.167659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:12.167659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:24:12.167659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:12.167659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:12.167659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:12.167659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:12.167659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:12.167659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:12.167659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:12.167659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:12.167659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:12.167659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:12.167659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:12.167659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:12.168527 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 20 13:24:12.168527 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 20 13:24:12.168527 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:12.168527 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:12.168527 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 20 13:24:12.168527 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 20 13:24:12.168527 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:12.168527 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:12.168527 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 20 13:24:12.168527 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 20 13:24:12.168527 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 20 13:24:12.168527 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:12.168527 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 406/806] 10.222.0.23 () {60 vars in 1137 bytes} [Thu Oct 20 13:24:11 2022] POST /identity/v3/domains => generated 190 bytes in 168 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:24:12.169629 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c689d5-37f3-4657-8ac7-ffea482777a7 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:12.210316 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb0b70-80a2-4eb2-9f88-cadf935e7fac None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:12.239341 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:12.277559 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c689d5-37f3-4657-8ac7-ffea482777a7 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:12.318375 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb0dca-3976-4e46-9fba-a5b2db6890eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:12.319315 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb0dca-3976-4e46-9fba-a5b2db6890eb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:12.319733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb0dca-3976-4e46-9fba-a5b2db6890eb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:12.320193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb0dca-3976-4e46-9fba-a5b2db6890eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:12.332683 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb0dca-3976-4e46-9fba-a5b2db6890eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:12.334456 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb0dca-3976-4e46-9fba-a5b2db6890eb None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:12.350769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-74fb0dca-3976-4e46-9fba-a5b2db6890eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:24:12.350769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:12.350769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:12.350769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:12.350769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:12.350769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:12.350769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:12.350769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:12.350769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:12.350769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:12.350769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:12.350769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:12.350769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:24:12.350769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:24:12.350769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:24:12.350769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:24:12.351840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:12.351840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:12.351840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:12.351840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:12.351840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:12.351840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:12.351840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:12.351840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:12.351840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:12.351840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:12.351840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:12.351840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:12.351840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:12.351840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:12.351840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:12.351840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:12.351840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:24:12.352727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:24:12.352727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:24:12.352727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:24:12.352727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 401/807] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:24:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:24:12.358997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb0dca-3976-4e46-9fba-a5b2db6890eb None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:12.438903 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb0b70-80a2-4eb2-9f88-cadf935e7fac None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:12.471618 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb0dca-3976-4e46-9fba-a5b2db6890eb None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:12.473347 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaacdb7-0676-42e2-a102-dd260e9c2279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:12.473979 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaacdb7-0676-42e2-a102-dd260e9c2279 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:12.474258 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaacdb7-0676-42e2-a102-dd260e9c2279 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:12.474537 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaacdb7-0676-42e2-a102-dd260e9c2279 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:12.475626 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaacdb7-0676-42e2-a102-dd260e9c2279 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:12.477102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaacdb7-0676-42e2-a102-dd260e9c2279 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:12.478677 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 407/808] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:24:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:24:12.484178 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaacdb7-0676-42e2-a102-dd260e9c2279 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:12.498007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:12.514882 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382e600-7464-4e8b-a556-7e02c0eb9830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:12.515497 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382e600-7464-4e8b-a556-7e02c0eb9830 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:12.515779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382e600-7464-4e8b-a556-7e02c0eb9830 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:12.516049 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382e600-7464-4e8b-a556-7e02c0eb9830 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:12.516691 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382e600-7464-4e8b-a556-7e02c0eb9830 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:12.517166 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382e600-7464-4e8b-a556-7e02c0eb9830 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:12.519740 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382e600-7464-4e8b-a556-7e02c0eb9830 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:12.533161 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 402/809] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:24:12.550413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:12.610388 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaacdb7-0676-42e2-a102-dd260e9c2279 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:12.647719 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8b668-c45d-4a52-b53a-8000c1bb00d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:12.648340 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8b668-c45d-4a52-b53a-8000c1bb00d6 None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:12.648618 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8b668-c45d-4a52-b53a-8000c1bb00d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:12.648913 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8b668-c45d-4a52-b53a-8000c1bb00d6 None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:12.649578 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8b668-c45d-4a52-b53a-8000c1bb00d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:12.651075 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8b668-c45d-4a52-b53a-8000c1bb00d6 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:12.674730 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 408/810] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 20 13:24:12 2022] POST /identity/v3/users => generated 285 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:24:13.975903 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662d295-c36a-4693-b411-d466f08aea0f None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:13.976463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662d295-c36a-4693-b411-d466f08aea0f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:13.976463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662d295-c36a-4693-b411-d466f08aea0f None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:13.977795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 403/811] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:24:13.983137 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7eca2-b2bd-4af7-84a1-b77ed92eda07 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:13.983523 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7eca2-b2bd-4af7-84a1-b77ed92eda07 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:13.983686 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7eca2-b2bd-4af7-84a1-b77ed92eda07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:14.001863 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-3de7eca2-b2bd-4af7-84a1-b77ed92eda07 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:14.003644 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-3de7eca2-b2bd-4af7-84a1-b77ed92eda07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:14.067284 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 409/812] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:24:14.334694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662d295-c36a-4693-b411-d466f08aea0f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:14.438125 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662d295-c36a-4693-b411-d466f08aea0f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:14.472610 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52f8f8-ea8a-43aa-837e-f46fec696f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:14.473272 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52f8f8-ea8a-43aa-837e-f46fec696f62 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:14.473556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52f8f8-ea8a-43aa-837e-f46fec696f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:14.473831 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52f8f8-ea8a-43aa-837e-f46fec696f62 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:14.483820 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52f8f8-ea8a-43aa-837e-f46fec696f62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:24:14.489795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-1c52f8f8-ea8a-43aa-837e-f46fec696f62 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 20 13:24:14.489795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:14.489795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:14.489795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:14.489795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:14.489795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:14.489795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:14.489795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:14.489795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:14.489795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:14.489795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:14.489795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:14.489795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:24:14.489795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:24:14.489795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:24:14.489795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:24:14.490878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:24:14.490878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:24:14.490878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:24:14.490878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:24:14.490878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:14.490878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:14.490878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:14.490878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:14.490878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:14.490878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:14.490878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:14.490878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:14.490878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:14.490878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:14.490878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:14.490878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:14.491962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:14.491962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:14.491962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:14.491962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:14.491962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:24:14.491962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:24:14.491962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:24:14.491962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:24:14.491962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:24:14.491962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:24:14.491962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 20 13:24:14.491962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:24:14.493021 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 404/813] 10.222.0.23 () {56 vars in 1120 bytes} [Thu Oct 20 13:24:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:24:14.498861 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7eca2-b2bd-4af7-84a1-b77ed92eda07 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:14.532310 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:14.596848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7eca2-b2bd-4af7-84a1-b77ed92eda07 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:14.628513 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6826c5-d18d-4112-84c3-3ae6071c6263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:14.629135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6826c5-d18d-4112-84c3-3ae6071c6263 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:14.629411 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6826c5-d18d-4112-84c3-3ae6071c6263 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:14.629680 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6826c5-d18d-4112-84c3-3ae6071c6263 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:14.630329 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6826c5-d18d-4112-84c3-3ae6071c6263 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:14.630626 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6826c5-d18d-4112-84c3-3ae6071c6263 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:14.631992 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6826c5-d18d-4112-84c3-3ae6071c6263 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:14.639749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 410/814] 10.222.0.23 () {56 vars in 1115 bytes} [Thu Oct 20 13:24:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:14.646091 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52f8f8-ea8a-43aa-837e-f46fec696f62 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:14.688405 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:14.747093 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52f8f8-ea8a-43aa-837e-f46fec696f62 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:14.783272 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a27cab-2512-4332-a90e-3f321e3fc64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:14.784062 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a27cab-2512-4332-a90e-3f321e3fc64a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:14.784342 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a27cab-2512-4332-a90e-3f321e3fc64a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:14.784615 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a27cab-2512-4332-a90e-3f321e3fc64a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:14.794443 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a27cab-2512-4332-a90e-3f321e3fc64a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:14.795875 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a27cab-2512-4332-a90e-3f321e3fc64a None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:14.810750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-90a27cab-2512-4332-a90e-3f321e3fc64a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:24:14.810750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:14.810750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:14.810750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:14.810750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:14.810750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:14.810750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:14.810750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:14.810750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:14.810750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:14.810750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:14.810750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:14.810750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:24:14.810750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:24:14.810750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:24:14.810750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:24:14.811806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:14.811806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:14.811806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:14.811806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:14.811806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:14.811806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:14.811806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:14.811806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:14.811806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:14.811806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:14.811806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:14.811806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:14.811806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:14.811806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:14.811806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:14.811806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:14.811806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:24:14.812699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:24:14.812699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:24:14.812699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:24:14.812699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 405/815] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:24:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:24:14.818125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6826c5-d18d-4112-84c3-3ae6071c6263 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:14.846656 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11830537-850d-4841-85a2-ffe30c66c60d None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:14.846992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11830537-850d-4841-85a2-ffe30c66c60d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:14.847220 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11830537-850d-4841-85a2-ffe30c66c60d None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:14.848765 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 406/816] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:24:14.853706 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d05737-f50d-46c3-8ee5-486e004e2794 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:14.853985 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d05737-f50d-46c3-8ee5-486e004e2794 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:14.854252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d05737-f50d-46c3-8ee5-486e004e2794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:14.872263 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-09d05737-f50d-46c3-8ee5-486e004e2794 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:14.873460 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-09d05737-f50d-46c3-8ee5-486e004e2794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:14.913217 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:14.917310 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6826c5-d18d-4112-84c3-3ae6071c6263 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:14.942027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 407/817] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 13:24:14.950943 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485b52e-7554-41f9-bed1-fb030dc143ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:14.951545 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485b52e-7554-41f9-bed1-fb030dc143ec None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:14.951823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485b52e-7554-41f9-bed1-fb030dc143ec None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:14.952092 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485b52e-7554-41f9-bed1-fb030dc143ec None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:14.952723 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485b52e-7554-41f9-bed1-fb030dc143ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:14.953051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485b52e-7554-41f9-bed1-fb030dc143ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:14.954413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485b52e-7554-41f9-bed1-fb030dc143ec None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:14.965876 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 411/818] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:14.972045 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d05737-f50d-46c3-8ee5-486e004e2794 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:14.995795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:15.068011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d05737-f50d-46c3-8ee5-486e004e2794 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:15.100822 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36d91b-45cd-4699-a890-ff78a6d33cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:15.101401 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36d91b-45cd-4699-a890-ff78a6d33cdb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:15.101679 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36d91b-45cd-4699-a890-ff78a6d33cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:15.101948 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36d91b-45cd-4699-a890-ff78a6d33cdb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:15.108663 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36d91b-45cd-4699-a890-ff78a6d33cdb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:15.110137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36d91b-45cd-4699-a890-ff78a6d33cdb None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:15.122532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-ab36d91b-45cd-4699-a890-ff78a6d33cdb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 20 13:24:15.122532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:15.122532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:15.122532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:15.122532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:15.122532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:15.122532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:15.122532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:15.122532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:15.122532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:15.122532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:15.122532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:15.122532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:24:15.122532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:24:15.122532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:24:15.122532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:24:15.123466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:15.123466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:15.123466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:24:15.123466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:24:15.123466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:24:15.123466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:24:15.123466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:15.123466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:15.123466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:15.123466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:15.123466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:15.123466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:15.123466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:15.123466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:15.123466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:15.123466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:15.123466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:15.123466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:15.124404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:15.124404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:15.124404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:24:15.124404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:24:15.124404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:24:15.124404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:24:15.124404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:24:15.124404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:24:15.124404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 20 13:24:15.124404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:24:15.124404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 408/819] 10.222.0.23 () {56 vars in 1093 bytes} [Thu Oct 20 13:24:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:24:15.129841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485b52e-7554-41f9-bed1-fb030dc143ec None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:15.212332 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36d91b-45cd-4699-a890-ff78a6d33cdb None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:15.228133 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485b52e-7554-41f9-bed1-fb030dc143ec None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:15.263043 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6ce0f-5941-468c-8117-18285adbcb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:15.263639 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6ce0f-5941-468c-8117-18285adbcb20 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:15.263918 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6ce0f-5941-468c-8117-18285adbcb20 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:15.264190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6ce0f-5941-468c-8117-18285adbcb20 None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:15.264937 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc6ce0f-5941-468c-8117-18285adbcb20 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:15.265242 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc6ce0f-5941-468c-8117-18285adbcb20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:15.266665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc6ce0f-5941-468c-8117-18285adbcb20 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:15.285397 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 412/820] 10.222.0.23 () {56 vars in 1133 bytes} [Thu Oct 20 13:24:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:15.291044 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6ce0f-5941-468c-8117-18285adbcb20 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:15.293899 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:15.308242 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:15.312375 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36d91b-45cd-4699-a890-ff78a6d33cdb None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:15.345121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053197b-630c-4992-9aa8-4055eb0953b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:15.345764 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053197b-630c-4992-9aa8-4055eb0953b0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:15.346179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053197b-630c-4992-9aa8-4055eb0953b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:15.346634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053197b-630c-4992-9aa8-4055eb0953b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:15.356898 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d053197b-630c-4992-9aa8-4055eb0953b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:24:15.362133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d053197b-630c-4992-9aa8-4055eb0953b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:24:15.362133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:15.362133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:15.362133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:15.362133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:15.362133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:15.362133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:15.362133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:15.362133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:15.362133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:15.362133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:15.362133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:15.362133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:24:15.362133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:24:15.362133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:24:15.362133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:24:15.363042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:24:15.363042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:24:15.363042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:24:15.363042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:24:15.363042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:15.363042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:15.363042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:15.363042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:15.363042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:15.363042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:15.363042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:15.363042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:15.363042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:15.363042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:15.363042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:15.363042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:15.364097 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:15.364097 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:15.364097 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:15.364097 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:15.364097 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:24:15.364097 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:24:15.364097 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:24:15.364097 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:24:15.364097 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:24:15.364097 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:24:15.364097 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:24:15.364097 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:24:15.364097 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 409/821] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 20 13:24:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:24:15.369091 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053197b-630c-4992-9aa8-4055eb0953b0 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:15.387837 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6ce0f-5941-468c-8117-18285adbcb20 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:15.420615 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b910f8-9d36-4cf1-89ed-5692f4e2f115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:15.421240 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b910f8-9d36-4cf1-89ed-5692f4e2f115 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:15.421521 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b910f8-9d36-4cf1-89ed-5692f4e2f115 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:15.421793 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b910f8-9d36-4cf1-89ed-5692f4e2f115 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:15.468621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053197b-630c-4992-9aa8-4055eb0953b0 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:15.489420 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b910f8-9d36-4cf1-89ed-5692f4e2f115 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:15.490893 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b910f8-9d36-4cf1-89ed-5692f4e2f115 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:15.503428 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507cabe-7dc9-4840-91a0-6149d68be5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:15.504039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507cabe-7dc9-4840-91a0-6149d68be5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:15.504443 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507cabe-7dc9-4840-91a0-6149d68be5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:15.504723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507cabe-7dc9-4840-91a0-6149d68be5d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:15.505545 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507cabe-7dc9-4840-91a0-6149d68be5d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:15.505852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507cabe-7dc9-4840-91a0-6149d68be5d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:15.507244 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507cabe-7dc9-4840-91a0-6149d68be5d4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:15.507571 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:15.516720 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 410/822] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:24:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:24:15.523019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507cabe-7dc9-4840-91a0-6149d68be5d4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:15.526518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:15.588834 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 413/823] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:24:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 20 13:24:15.594573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b910f8-9d36-4cf1-89ed-5692f4e2f115 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:15.625065 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507cabe-7dc9-4840-91a0-6149d68be5d4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:15.663903 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff2a6b-fc85-40ef-96f5-44cc3db33964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:15.664527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff2a6b-fc85-40ef-96f5-44cc3db33964 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:15.664807 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff2a6b-fc85-40ef-96f5-44cc3db33964 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:15.665109 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff2a6b-fc85-40ef-96f5-44cc3db33964 None admin] PATH_INFO: `/v3/users/ca39424ab9e041f7888336b488c7d582` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:15.678875 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff2a6b-fc85-40ef-96f5-44cc3db33964 None admin] RBAC: Authorizing `identity:get_user(user_id=ca39424ab9e041f7888336b488c7d582)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:15.680331 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff2a6b-fc85-40ef-96f5-44cc3db33964 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:15.681971 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 411/824] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:24:15 2022] GET /identity/v3/users/ca39424ab9e041f7888336b488c7d582 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:24:15.687794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff2a6b-fc85-40ef-96f5-44cc3db33964 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:15.690331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b910f8-9d36-4cf1-89ed-5692f4e2f115 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:15.707636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:15.725982 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a207b2a-2ecd-42ba-a1c5-9d3fdc025967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:15.726611 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a207b2a-2ecd-42ba-a1c5-9d3fdc025967 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:15.726892 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a207b2a-2ecd-42ba-a1c5-9d3fdc025967 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:15.727163 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a207b2a-2ecd-42ba-a1c5-9d3fdc025967 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:15.737485 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a207b2a-2ecd-42ba-a1c5-9d3fdc025967 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:15.738897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a207b2a-2ecd-42ba-a1c5-9d3fdc025967 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:15.748401 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:15.753881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-3a207b2a-2ecd-42ba-a1c5-9d3fdc025967 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:24:15.753881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:15.753881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:15.753881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:15.753881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:15.753881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:15.753881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:15.753881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:15.753881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:15.753881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:15.753881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:15.753881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:15.753881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:24:15.753881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:24:15.753881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:24:15.753881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:24:15.754987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:15.754987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:15.754987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:15.754987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:15.754987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:15.754987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:15.754987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:15.754987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:15.754987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:15.754987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:15.754987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:15.754987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:15.754987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:15.754987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:15.754987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:15.754987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:15.754987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:24:15.755871 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:24:15.755871 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:24:15.755871 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:15.755871 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 414/825] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:24:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:24:15.760534 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a207b2a-2ecd-42ba-a1c5-9d3fdc025967 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:15.789920 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff2a6b-fc85-40ef-96f5-44cc3db33964 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:15.823772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffd429-7db1-498c-b9bb-a3e5d10cb64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:15.824395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffd429-7db1-498c-b9bb-a3e5d10cb64e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:15.824676 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffd429-7db1-498c-b9bb-a3e5d10cb64e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:15.824973 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffd429-7db1-498c-b9bb-a3e5d10cb64e None admin] PATH_INFO: `/v3/projects/405579f7326f46849acfee731877df25` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:15.826078 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ffd429-7db1-498c-b9bb-a3e5d10cb64e None admin] RBAC: Authorizing `identity:get_project(project_id=405579f7326f46849acfee731877df25)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:15.827556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ffd429-7db1-498c-b9bb-a3e5d10cb64e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:15.829206 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 412/826] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:24:15 2022] GET /identity/v3/projects/405579f7326f46849acfee731877df25 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:24:15.835805 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffd429-7db1-498c-b9bb-a3e5d10cb64e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:15.858548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a207b2a-2ecd-42ba-a1c5-9d3fdc025967 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:15.891708 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbf964-2381-435d-a3e0-c391e888f2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:15.892243 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbf964-2381-435d-a3e0-c391e888f2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:15.892524 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbf964-2381-435d-a3e0-c391e888f2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:15.893517 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dbf964-2381-435d-a3e0-c391e888f2f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:15.894317 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbf964-2381-435d-a3e0-c391e888f2f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:15.894621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbf964-2381-435d-a3e0-c391e888f2f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:15.895996 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dbf964-2381-435d-a3e0-c391e888f2f1 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:15.908143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 415/827] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:24:15.913645 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbf964-2381-435d-a3e0-c391e888f2f1 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:15.932906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffd429-7db1-498c-b9bb-a3e5d10cb64e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:15.940356 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:15.961081 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:15.967122 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02060c5b-3744-4057-8ef5-db5b0e446570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:15.967783 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02060c5b-3744-4057-8ef5-db5b0e446570 None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:15.968065 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02060c5b-3744-4057-8ef5-db5b0e446570 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:15.968351 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02060c5b-3744-4057-8ef5-db5b0e446570 None admin] PATH_INFO: `/v3/projects/405579f7326f46849acfee731877df25/users/ca39424ab9e041f7888336b488c7d582/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:15.970282 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02060c5b-3744-4057-8ef5-db5b0e446570 None admin] RBAC: Authorizing `identity:create_grant(project_id=405579f7326f46849acfee731877df25, user_id=ca39424ab9e041f7888336b488c7d582, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:15.972356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02060c5b-3744-4057-8ef5-db5b0e446570 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:15.984389 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 413/828] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:24:15 2022] PUT /identity/v3/projects/405579f7326f46849acfee731877df25/users/ca39424ab9e041f7888336b488c7d582/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:24:16.014105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dbf964-2381-435d-a3e0-c391e888f2f1 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:16.051593 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f489617-b119-4fd9-9a3f-7f54540d329c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:16.052235 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f489617-b119-4fd9-9a3f-7f54540d329c None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:16.052512 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f489617-b119-4fd9-9a3f-7f54540d329c None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:16.052781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f489617-b119-4fd9-9a3f-7f54540d329c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:16.060138 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f489617-b119-4fd9-9a3f-7f54540d329c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:16.061645 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f489617-b119-4fd9-9a3f-7f54540d329c None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:16.074225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-1f489617-b119-4fd9-9a3f-7f54540d329c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:24:16.074225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:16.074225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:16.074225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:16.074225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:16.074225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:16.074225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:16.074225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:16.074225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:16.074225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:16.074225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:16.074225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:16.074225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:24:16.074225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:24:16.074225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:24:16.075331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:24:16.075331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:16.075331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:16.075331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:16.075331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:16.075331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:16.075331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:16.075331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:16.075331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:16.075331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:16.075331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:16.075331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:16.075331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:16.075331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:16.075331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:16.075331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:16.075331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:16.076651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:24:16.076651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:24:16.076651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:24:16.076651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:24:16.076651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:24:16.076651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:24:16.076651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:24:16.076651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:16.076651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 416/829] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:24:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:24:16.082223 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02060c5b-3744-4057-8ef5-db5b0e446570 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:16.186492 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02060c5b-3744-4057-8ef5-db5b0e446570 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:16.219163 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f90bb5-18ae-411a-81e8-8851d81ec665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:16.219762 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f90bb5-18ae-411a-81e8-8851d81ec665 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:16.220038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f90bb5-18ae-411a-81e8-8851d81ec665 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:16.220307 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f90bb5-18ae-411a-81e8-8851d81ec665 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:16.221028 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f90bb5-18ae-411a-81e8-8851d81ec665 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:16.221361 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f90bb5-18ae-411a-81e8-8851d81ec665 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:16.222774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f90bb5-18ae-411a-81e8-8851d81ec665 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:16.234199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 414/830] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 20 13:24:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:16.240826 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f489617-b119-4fd9-9a3f-7f54540d329c None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:16.327968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:16.340759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f489617-b119-4fd9-9a3f-7f54540d329c None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:16.374252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1a5f1-28d6-48ae-a0cb-197e2a72b2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:16.374845 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1a5f1-28d6-48ae-a0cb-197e2a72b2a2 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:16.375118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1a5f1-28d6-48ae-a0cb-197e2a72b2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:16.375385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1a5f1-28d6-48ae-a0cb-197e2a72b2a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:16.376098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb1a5f1-28d6-48ae-a0cb-197e2a72b2a2 None admin] RBAC: Adding query filter params (role.id=ae10ab7a1f7f4eddae7ca60cc47d60d1, scope.project.id=4a81b6ecfb6e46a8b08c8bc2b4000fc7, user.id=ee5c2a3855cb4a668177aa19f337778d) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:16.376399 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb1a5f1-28d6-48ae-a0cb-197e2a72b2a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:16.377920 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb1a5f1-28d6-48ae-a0cb-197e2a72b2a2 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:16.383713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 417/831] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:24:16 2022] GET /identity/v3/role_assignments?user.id=ee5c2a3855cb4a668177aa19f337778d&scope.project.id=4a81b6ecfb6e46a8b08c8bc2b4000fc7&role.id=ae10ab7a1f7f4eddae7ca60cc47d60d1 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:18.114157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39945b6-3ea9-41b3-855c-16636a01a7e1 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:18.114688 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39945b6-3ea9-41b3-855c-16636a01a7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:18.114688 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39945b6-3ea9-41b3-855c-16636a01a7e1 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:18.116038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 415/832] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:24:18.121231 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4755b-bdfc-4091-ab77-849a38ba11a9 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:18.121975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4755b-bdfc-4091-ab77-849a38ba11a9 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:18.122249 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4755b-bdfc-4091-ab77-849a38ba11a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:18.140940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-0df4755b-bdfc-4091-ab77-849a38ba11a9 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:18.142128 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-0df4755b-bdfc-4091-ab77-849a38ba11a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:18.206142 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 418/833] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 13:24:18.478292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39945b6-3ea9-41b3-855c-16636a01a7e1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:18.529023 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:18.579535 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39945b6-3ea9-41b3-855c-16636a01a7e1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:18.613363 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed487254-96cb-4509-9298-9c486bf53212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:18.613981 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed487254-96cb-4509-9298-9c486bf53212 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:18.614262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed487254-96cb-4509-9298-9c486bf53212 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:18.614535 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed487254-96cb-4509-9298-9c486bf53212 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:18.625291 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed487254-96cb-4509-9298-9c486bf53212 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:24:18.630480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-ed487254-96cb-4509-9298-9c486bf53212 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:24:18.630480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:18.630480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:18.630480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:18.630480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:18.630480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:18.630480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:18.630480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:18.630480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:18.630480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:18.630480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:18.630480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:18.630480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:24:18.630480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:24:18.630480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:24:18.630480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:24:18.631529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:24:18.631529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:24:18.631529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:24:18.631529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:24:18.631529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:18.631529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:18.631529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:18.631529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:18.631529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:18.631529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:18.631529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:18.631529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:18.631529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:18.631529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:18.631529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:18.631529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:18.633792 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:18.633792 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:18.633792 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:18.633792 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:18.633792 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:24:18.633792 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:24:18.633792 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:24:18.633792 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:24:18.633792 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:24:18.633792 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:24:18.633792 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:24:18.633792 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:24:18.633792 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 416/834] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 20 13:24:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:24:18.638040 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4755b-bdfc-4091-ab77-849a38ba11a9 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:18.715464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:18.733683 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4755b-bdfc-4091-ab77-849a38ba11a9 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:18.766302 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d698fdd-454e-4cde-8d85-39ffa2768902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:18.766898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d698fdd-454e-4cde-8d85-39ffa2768902 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:18.767178 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d698fdd-454e-4cde-8d85-39ffa2768902 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:18.767486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d698fdd-454e-4cde-8d85-39ffa2768902 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:18.768188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d698fdd-454e-4cde-8d85-39ffa2768902 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:18.768489 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d698fdd-454e-4cde-8d85-39ffa2768902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:18.769884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d698fdd-454e-4cde-8d85-39ffa2768902 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:18.777560 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 419/835] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:24:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:24:18.784036 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed487254-96cb-4509-9298-9c486bf53212 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:18.878894 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed487254-96cb-4509-9298-9c486bf53212 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:18.888754 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:18.911495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac15aaf-b72a-46d6-8fa7-18fe45ceed59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:18.912099 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac15aaf-b72a-46d6-8fa7-18fe45ceed59 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:18.912376 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac15aaf-b72a-46d6-8fa7-18fe45ceed59 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:18.912647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac15aaf-b72a-46d6-8fa7-18fe45ceed59 None admin] PATH_INFO: `/v3/users/ca39424ab9e041f7888336b488c7d582` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:18.913765 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac15aaf-b72a-46d6-8fa7-18fe45ceed59 None admin] RBAC: Authorizing `identity:get_user(user_id=ca39424ab9e041f7888336b488c7d582)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:18.915166 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac15aaf-b72a-46d6-8fa7-18fe45ceed59 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:18.916693 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 417/836] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:24:18 2022] GET /identity/v3/users/ca39424ab9e041f7888336b488c7d582 => generated 285 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:18.922285 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d698fdd-454e-4cde-8d85-39ffa2768902 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:19.017404 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d698fdd-454e-4cde-8d85-39ffa2768902 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:19.049823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fdd7ea-56c5-4f65-90cc-53b9fe465b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:19.050413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdd7ea-56c5-4f65-90cc-53b9fe465b67 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:19.050691 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdd7ea-56c5-4f65-90cc-53b9fe465b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:19.050961 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdd7ea-56c5-4f65-90cc-53b9fe465b67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:19.065302 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:19.116687 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fdd7ea-56c5-4f65-90cc-53b9fe465b67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:19.118142 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fdd7ea-56c5-4f65-90cc-53b9fe465b67 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:19.210879 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 420/837] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:24:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 13:24:19.216781 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac15aaf-b72a-46d6-8fa7-18fe45ceed59 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:19.312158 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac15aaf-b72a-46d6-8fa7-18fe45ceed59 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:19.344280 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdaac8-1b7a-45d9-b525-8b01e0cba8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:19.344910 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdaac8-1b7a-45d9-b525-8b01e0cba8f0 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:19.345194 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdaac8-1b7a-45d9-b525-8b01e0cba8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:19.345465 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdaac8-1b7a-45d9-b525-8b01e0cba8f0 None admin] PATH_INFO: `/v3/projects/405579f7326f46849acfee731877df25` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:19.346758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdaac8-1b7a-45d9-b525-8b01e0cba8f0 None admin] RBAC: Authorizing `identity:get_project(project_id=405579f7326f46849acfee731877df25)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:19.348178 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdaac8-1b7a-45d9-b525-8b01e0cba8f0 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:19.349786 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 418/838] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:24:19 2022] GET /identity/v3/projects/405579f7326f46849acfee731877df25 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:19.355799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fdd7ea-56c5-4f65-90cc-53b9fe465b67 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:19.399780 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:19.451322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fdd7ea-56c5-4f65-90cc-53b9fe465b67 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:19.484324 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3165d5-a6ae-465b-8ae6-452f0ffbda29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:19.484942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3165d5-a6ae-465b-8ae6-452f0ffbda29 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:19.485222 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3165d5-a6ae-465b-8ae6-452f0ffbda29 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:19.485491 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3165d5-a6ae-465b-8ae6-452f0ffbda29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:19.486226 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3165d5-a6ae-465b-8ae6-452f0ffbda29 None admin] RBAC: Adding query filter params (role.id=2bcf35e1bc5f4634b0aa6411fad2f43f, scope.project.id=405579f7326f46849acfee731877df25, user.id=ca39424ab9e041f7888336b488c7d582) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:19.486532 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3165d5-a6ae-465b-8ae6-452f0ffbda29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:19.487943 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3165d5-a6ae-465b-8ae6-452f0ffbda29 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:19.493541 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 421/839] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:24:19 2022] GET /identity/v3/role_assignments?user.id=ca39424ab9e041f7888336b488c7d582&scope.project.id=405579f7326f46849acfee731877df25&role.id=2bcf35e1bc5f4634b0aa6411fad2f43f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:21.611852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4857d5c-0228-4770-9ad6-7dae38dafae0 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:21.612370 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4857d5c-0228-4770-9ad6-7dae38dafae0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:21.612518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4857d5c-0228-4770-9ad6-7dae38dafae0 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:21.613908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 419/840] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:24:21.618976 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47fa63-0f4f-4bf6-be04-d9a596121c2d None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:21.619411 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47fa63-0f4f-4bf6-be04-d9a596121c2d None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:21.619502 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47fa63-0f4f-4bf6-be04-d9a596121c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:21.637897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-de47fa63-0f4f-4bf6-be04-d9a596121c2d None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:21.639060 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-de47fa63-0f4f-4bf6-be04-d9a596121c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:21.702860 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 422/841] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:24:21.969067 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4857d5c-0228-4770-9ad6-7dae38dafae0 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:22.015908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:22.068193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4857d5c-0228-4770-9ad6-7dae38dafae0 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:22.102181 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55d4aa-0683-4965-8d6d-93ce2c2f6c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:22.102794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55d4aa-0683-4965-8d6d-93ce2c2f6c57 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:22.103075 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55d4aa-0683-4965-8d6d-93ce2c2f6c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:22.103367 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55d4aa-0683-4965-8d6d-93ce2c2f6c57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:22.104456 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc55d4aa-0683-4965-8d6d-93ce2c2f6c57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:22.105912 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc55d4aa-0683-4965-8d6d-93ce2c2f6c57 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:22.107450 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 420/842] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:24:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:22.113261 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47fa63-0f4f-4bf6-be04-d9a596121c2d None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:22.151078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:22.208573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47fa63-0f4f-4bf6-be04-d9a596121c2d None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:22.241114 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716940e-8ec6-4be6-bba6-757eb82e11c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:22.241649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716940e-8ec6-4be6-bba6-757eb82e11c4 None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:22.241924 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716940e-8ec6-4be6-bba6-757eb82e11c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:22.242190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716940e-8ec6-4be6-bba6-757eb82e11c4 None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:22.242819 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8716940e-8ec6-4be6-bba6-757eb82e11c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:22.244234 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8716940e-8ec6-4be6-bba6-757eb82e11c4 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:22.265679 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 423/843] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 20 13:24:22 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:24:24.387920 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1b07e-a0c4-45d2-bd33-cb0ef8fb716a None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:24.388452 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1b07e-a0c4-45d2-bd33-cb0ef8fb716a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:24.388452 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1b07e-a0c4-45d2-bd33-cb0ef8fb716a None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:24.389840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 421/844] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:24:24.395783 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4ca02-9ad4-4809-88bd-3e95fbb19814 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:24.396212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4ca02-9ad4-4809-88bd-3e95fbb19814 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:24.396326 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4ca02-9ad4-4809-88bd-3e95fbb19814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:24.414503 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-3ce4ca02-9ad4-4809-88bd-3e95fbb19814 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:24.415662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-3ce4ca02-9ad4-4809-88bd-3e95fbb19814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:24.478969 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 424/845] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:24:24.749563 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1b07e-a0c4-45d2-bd33-cb0ef8fb716a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:24.846755 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1b07e-a0c4-45d2-bd33-cb0ef8fb716a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:24.859749 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:24.879374 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2ef59-ed83-48e2-9dd9-c5cc0925fa0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:24.879974 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2ef59-ed83-48e2-9dd9-c5cc0925fa0c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:24.880250 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2ef59-ed83-48e2-9dd9-c5cc0925fa0c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:24.880518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d2ef59-ed83-48e2-9dd9-c5cc0925fa0c None admin] PATH_INFO: `/v3/roles/b0ccf031da9e4a7492c567b607ceb21f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:24.881881 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d2ef59-ed83-48e2-9dd9-c5cc0925fa0c None admin] RBAC: Authorizing `identity:get_role(role_id=b0ccf031da9e4a7492c567b607ceb21f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:24.883224 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d2ef59-ed83-48e2-9dd9-c5cc0925fa0c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:24.884507 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 422/846] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:24:24 2022] GET /identity/v3/roles/b0ccf031da9e4a7492c567b607ceb21f => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:24.890524 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4ca02-9ad4-4809-88bd-3e95fbb19814 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:24.914210 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:24.987779 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4ca02-9ad4-4809-88bd-3e95fbb19814 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:25.022521 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2146ed0e-dc1e-4e3c-9b72-ee5f759e0f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:25.023081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146ed0e-dc1e-4e3c-9b72-ee5f759e0f79 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:25.023360 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146ed0e-dc1e-4e3c-9b72-ee5f759e0f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:25.023632 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146ed0e-dc1e-4e3c-9b72-ee5f759e0f79 None admin] PATH_INFO: `/v3/users/00f436e199f7400190f024a125514a03` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:25.036486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2146ed0e-dc1e-4e3c-9b72-ee5f759e0f79 None admin] RBAC: Authorizing `identity:get_user(user_id=00f436e199f7400190f024a125514a03)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:25.037948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2146ed0e-dc1e-4e3c-9b72-ee5f759e0f79 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:25.039518 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 425/847] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:24:24 2022] GET /identity/v3/users/00f436e199f7400190f024a125514a03 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:25.045320 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2ef59-ed83-48e2-9dd9-c5cc0925fa0c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:25.145930 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d2ef59-ed83-48e2-9dd9-c5cc0925fa0c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:25.178524 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2927d-14af-4764-9a23-b8aef881a5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:25.179142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2927d-14af-4764-9a23-b8aef881a5db None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:25.179422 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2927d-14af-4764-9a23-b8aef881a5db None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:25.179694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2927d-14af-4764-9a23-b8aef881a5db None admin] PATH_INFO: `/v3/projects/405579f7326f46849acfee731877df25` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:25.180783 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b2927d-14af-4764-9a23-b8aef881a5db None admin] RBAC: Authorizing `identity:get_project(project_id=405579f7326f46849acfee731877df25)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:25.182254 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b2927d-14af-4764-9a23-b8aef881a5db None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:25.183854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 423/848] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:24:25 2022] GET /identity/v3/projects/405579f7326f46849acfee731877df25 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:25.190347 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2146ed0e-dc1e-4e3c-9b72-ee5f759e0f79 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:25.269126 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:25.286647 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2146ed0e-dc1e-4e3c-9b72-ee5f759e0f79 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:25.320574 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f26ce-74c0-4c3a-b8fa-e24f44489c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:25.321265 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f26ce-74c0-4c3a-b8fa-e24f44489c50 None admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:25.321531 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f26ce-74c0-4c3a-b8fa-e24f44489c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:25.321801 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f26ce-74c0-4c3a-b8fa-e24f44489c50 None admin] PATH_INFO: `/v3/projects/405579f7326f46849acfee731877df25/users/00f436e199f7400190f024a125514a03/roles/b0ccf031da9e4a7492c567b607ceb21f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:25.323900 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f26ce-74c0-4c3a-b8fa-e24f44489c50 None admin] RBAC: Authorizing `identity:create_grant(project_id=405579f7326f46849acfee731877df25, user_id=00f436e199f7400190f024a125514a03, role_id=b0ccf031da9e4a7492c567b607ceb21f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:25.325367 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f26ce-74c0-4c3a-b8fa-e24f44489c50 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:25.337447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 426/849] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:24:25 2022] PUT /identity/v3/projects/405579f7326f46849acfee731877df25/users/00f436e199f7400190f024a125514a03/roles/b0ccf031da9e4a7492c567b607ceb21f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:24:27.432965 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b9c25-acc4-4528-8cf4-10fbd6ced3f5 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:27.433512 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b9c25-acc4-4528-8cf4-10fbd6ced3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:27.433512 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b9c25-acc4-4528-8cf4-10fbd6ced3f5 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:27.434850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 424/850] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:24:27.439999 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16b8ea-a3a0-4946-9b8d-ddf6104426ee None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:27.440419 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16b8ea-a3a0-4946-9b8d-ddf6104426ee None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:27.440545 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16b8ea-a3a0-4946-9b8d-ddf6104426ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:27.458628 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-9b16b8ea-a3a0-4946-9b8d-ddf6104426ee None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:27.459780 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-9b16b8ea-a3a0-4946-9b8d-ddf6104426ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:27.523156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 427/851] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:24:27.789663 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b9c25-acc4-4528-8cf4-10fbd6ced3f5 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:27.839071 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:27.887862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b9c25-acc4-4528-8cf4-10fbd6ced3f5 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:27.919340 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28939cc-9479-486b-aeb9-7e0ae616307f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:27.919948 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28939cc-9479-486b-aeb9-7e0ae616307f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:27.920225 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28939cc-9479-486b-aeb9-7e0ae616307f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:27.920493 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28939cc-9479-486b-aeb9-7e0ae616307f None admin] PATH_INFO: `/v3/roles/b0ccf031da9e4a7492c567b607ceb21f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:27.921853 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28939cc-9479-486b-aeb9-7e0ae616307f None admin] RBAC: Authorizing `identity:get_role(role_id=b0ccf031da9e4a7492c567b607ceb21f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:27.923217 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28939cc-9479-486b-aeb9-7e0ae616307f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:27.924784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 425/852] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:24:27 2022] GET /identity/v3/roles/b0ccf031da9e4a7492c567b607ceb21f => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:27.930608 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16b8ea-a3a0-4946-9b8d-ddf6104426ee None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:27.997160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:28.027055 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16b8ea-a3a0-4946-9b8d-ddf6104426ee None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:28.059505 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0325ef-3168-4717-917f-438eeca2cdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:28.060129 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0325ef-3168-4717-917f-438eeca2cdec None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:28.060408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0325ef-3168-4717-917f-438eeca2cdec None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:28.060679 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0325ef-3168-4717-917f-438eeca2cdec None admin] PATH_INFO: `/v3/users/00f436e199f7400190f024a125514a03` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:28.061800 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0325ef-3168-4717-917f-438eeca2cdec None admin] RBAC: Authorizing `identity:get_user(user_id=00f436e199f7400190f024a125514a03)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:28.063224 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0325ef-3168-4717-917f-438eeca2cdec None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:28.064978 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 428/853] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:24:27 2022] GET /identity/v3/users/00f436e199f7400190f024a125514a03 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:28.070665 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28939cc-9479-486b-aeb9-7e0ae616307f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:28.168632 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28939cc-9479-486b-aeb9-7e0ae616307f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:28.184525 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:28.201575 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc8c52-5548-4906-b0a4-49e3d644e610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:28.202171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc8c52-5548-4906-b0a4-49e3d644e610 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:28.202455 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc8c52-5548-4906-b0a4-49e3d644e610 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:28.202727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc8c52-5548-4906-b0a4-49e3d644e610 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:28.267210 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc8c52-5548-4906-b0a4-49e3d644e610 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:28.269162 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc8c52-5548-4906-b0a4-49e3d644e610 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:28.363213 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 426/854] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:24:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 13:24:28.369104 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0325ef-3168-4717-917f-438eeca2cdec None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:28.464436 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0325ef-3168-4717-917f-438eeca2cdec None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:28.498300 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1be55-7a02-4cfa-94ab-cfe08d5598e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:28.498927 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1be55-7a02-4cfa-94ab-cfe08d5598e8 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:28.499207 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1be55-7a02-4cfa-94ab-cfe08d5598e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:28.499477 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1be55-7a02-4cfa-94ab-cfe08d5598e8 None admin] PATH_INFO: `/v3/projects/405579f7326f46849acfee731877df25` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:28.500581 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1be55-7a02-4cfa-94ab-cfe08d5598e8 None admin] RBAC: Authorizing `identity:get_project(project_id=405579f7326f46849acfee731877df25)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:28.502087 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1be55-7a02-4cfa-94ab-cfe08d5598e8 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:28.503708 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 429/855] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:24:28 2022] GET /identity/v3/projects/405579f7326f46849acfee731877df25 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:28.509987 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc8c52-5548-4906-b0a4-49e3d644e610 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:28.528898 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:28.605835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc8c52-5548-4906-b0a4-49e3d644e610 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:28.638405 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01f64c-29bb-4dd4-8429-038e671d65b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:28.638939 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01f64c-29bb-4dd4-8429-038e671d65b9 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:28.639213 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01f64c-29bb-4dd4-8429-038e671d65b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:28.639485 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01f64c-29bb-4dd4-8429-038e671d65b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:28.640193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01f64c-29bb-4dd4-8429-038e671d65b9 None admin] RBAC: Adding query filter params (role.id=b0ccf031da9e4a7492c567b607ceb21f, scope.project.id=405579f7326f46849acfee731877df25, user.id=00f436e199f7400190f024a125514a03) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:28.640498 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01f64c-29bb-4dd4-8429-038e671d65b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:28.641932 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01f64c-29bb-4dd4-8429-038e671d65b9 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:28.647407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 427/856] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:24:28 2022] GET /identity/v3/role_assignments?user.id=00f436e199f7400190f024a125514a03&scope.project.id=405579f7326f46849acfee731877df25&role.id=b0ccf031da9e4a7492c567b607ceb21f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:30.781083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d5258-accf-4822-bea0-12274951dbec None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:30.781579 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d5258-accf-4822-bea0-12274951dbec None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:30.781579 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d5258-accf-4822-bea0-12274951dbec None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:30.782908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 430/857] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:24:30.789005 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f816bc0-3ca0-435f-b6a6-87a41f797025 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:30.789373 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f816bc0-3ca0-435f-b6a6-87a41f797025 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:30.789557 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f816bc0-3ca0-435f-b6a6-87a41f797025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:30.808229 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5f816bc0-3ca0-435f-b6a6-87a41f797025 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:30.809446 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5f816bc0-3ca0-435f-b6a6-87a41f797025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:30.873712 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 428/858] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:24:31.149767 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d5258-accf-4822-bea0-12274951dbec None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:31.160659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:31.249190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d5258-accf-4822-bea0-12274951dbec None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:31.282550 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc609c43-a387-46cd-96e8-fa66013f5f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:31.283184 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc609c43-a387-46cd-96e8-fa66013f5f21 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:31.283840 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc609c43-a387-46cd-96e8-fa66013f5f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:31.284117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc609c43-a387-46cd-96e8-fa66013f5f21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:31.285254 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc609c43-a387-46cd-96e8-fa66013f5f21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:31.286691 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc609c43-a387-46cd-96e8-fa66013f5f21 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:31.288226 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 431/859] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:24:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:31.293952 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f816bc0-3ca0-435f-b6a6-87a41f797025 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:31.308147 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:31.390600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f816bc0-3ca0-435f-b6a6-87a41f797025 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:31.422883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d792c-ce67-46a4-a8e6-bca048364029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:31.423416 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d792c-ce67-46a4-a8e6-bca048364029 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:31.423693 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d792c-ce67-46a4-a8e6-bca048364029 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:31.424498 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d792c-ce67-46a4-a8e6-bca048364029 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:31.425171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d792c-ce67-46a4-a8e6-bca048364029 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:31.426588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d792c-ce67-46a4-a8e6-bca048364029 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:31.452532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 429/860] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 20 13:24:31 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:24:33.605356 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a142b3-81ef-4d63-8630-a1ccd44b0de1 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:33.605865 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a142b3-81ef-4d63-8630-a1ccd44b0de1 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:33.605952 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a142b3-81ef-4d63-8630-a1ccd44b0de1 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:33.607253 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 432/861] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:24:33.612431 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86515f-2917-429c-b735-ab495c45e348 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:33.612805 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86515f-2917-429c-b735-ab495c45e348 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:33.613022 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86515f-2917-429c-b735-ab495c45e348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:33.631576 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5b86515f-2917-429c-b735-ab495c45e348 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:33.632768 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5b86515f-2917-429c-b735-ab495c45e348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:33.804494 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:33.814206 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 430/862] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:24:34.084008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a142b3-81ef-4d63-8630-a1ccd44b0de1 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:34.160353 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:34.184040 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a142b3-81ef-4d63-8630-a1ccd44b0de1 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:34.217012 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3ad17-f493-4d2c-a82c-c8537b1249dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:34.217637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3ad17-f493-4d2c-a82c-c8537b1249dd None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:34.217916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3ad17-f493-4d2c-a82c-c8537b1249dd None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:34.218190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3ad17-f493-4d2c-a82c-c8537b1249dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:34.219288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e3ad17-f493-4d2c-a82c-c8537b1249dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:34.220745 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e3ad17-f493-4d2c-a82c-c8537b1249dd None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:34.222465 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 433/863] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:24:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:34.228110 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86515f-2917-429c-b735-ab495c45e348 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:34.321846 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86515f-2917-429c-b735-ab495c45e348 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:34.353070 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb8a051-d791-452a-8f8c-4a713eab2e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:34.353596 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8a051-d791-452a-8f8c-4a713eab2e71 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:34.353870 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8a051-d791-452a-8f8c-4a713eab2e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:34.354134 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8a051-d791-452a-8f8c-4a713eab2e71 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:34.354774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb8a051-d791-452a-8f8c-4a713eab2e71 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:34.356170 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb8a051-d791-452a-8f8c-4a713eab2e71 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:34.376660 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 431/864] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 20 13:24:34 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:24:36.548354 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d949330-fbfa-4094-87e3-f6431dca9278 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:36.548856 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d949330-fbfa-4094-87e3-f6431dca9278 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:36.548994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d949330-fbfa-4094-87e3-f6431dca9278 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:36.550330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 434/865] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:24:36.555530 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f9c4e-de3e-4b4e-8bf6-07eb1c038806 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:36.555924 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f9c4e-de3e-4b4e-8bf6-07eb1c038806 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:36.556087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f9c4e-de3e-4b4e-8bf6-07eb1c038806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:36.574050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0e7f9c4e-de3e-4b4e-8bf6-07eb1c038806 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:36.575216 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0e7f9c4e-de3e-4b4e-8bf6-07eb1c038806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:36.621576 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:36.642857 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 432/866] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:24:36.911262 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d949330-fbfa-4094-87e3-f6431dca9278 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:37.009520 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d949330-fbfa-4094-87e3-f6431dca9278 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:37.042717 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b9293-37ff-42c6-87ed-8506c1fca4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:37.043336 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b9293-37ff-42c6-87ed-8506c1fca4d4 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:37.043622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b9293-37ff-42c6-87ed-8506c1fca4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:37.043899 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b9293-37ff-42c6-87ed-8506c1fca4d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:37.054328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b9293-37ff-42c6-87ed-8506c1fca4d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:24:37.059134 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-958b9293-37ff-42c6-87ed-8506c1fca4d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:24:37.059134 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:37.059134 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:37.059134 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:37.059134 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:37.059134 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:37.059134 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:37.059134 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:37.059134 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:37.059134 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:37.059134 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:37.059134 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:37.059134 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:24:37.059134 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:24:37.059134 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:24:37.059134 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:24:37.060190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:24:37.060190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:24:37.060190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:24:37.060190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:24:37.060190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:37.060190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:37.060190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:37.060190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:37.060190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:37.060190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:37.060190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:37.060190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:37.060190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:37.060190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:37.060190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:37.060190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:37.061051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:37.061051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:37.061051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:37.061051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:37.061051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:24:37.061051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:24:37.061051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:24:37.061051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:24:37.061051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:24:37.061051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:24:37.061051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:24:37.061051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:37.061051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 435/867] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 20 13:24:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:24:37.066838 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f9c4e-de3e-4b4e-8bf6-07eb1c038806 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:37.166539 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f9c4e-de3e-4b4e-8bf6-07eb1c038806 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:37.198673 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c896315-392a-4c27-82e2-23540cb25d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:37.199263 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c896315-392a-4c27-82e2-23540cb25d95 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:37.199539 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c896315-392a-4c27-82e2-23540cb25d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:37.199808 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c896315-392a-4c27-82e2-23540cb25d95 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:37.200457 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c896315-392a-4c27-82e2-23540cb25d95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:37.200752 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c896315-392a-4c27-82e2-23540cb25d95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:37.202135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c896315-392a-4c27-82e2-23540cb25d95 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:37.209762 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 433/868] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:24:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:37.215561 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b9293-37ff-42c6-87ed-8506c1fca4d4 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:37.219811 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:37.312686 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b9293-37ff-42c6-87ed-8506c1fca4d4 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:37.346190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c540b-2d62-45b7-a88c-1238c81586a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:37.346799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c540b-2d62-45b7-a88c-1238c81586a4 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:37.347077 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c540b-2d62-45b7-a88c-1238c81586a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:37.347353 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c540b-2d62-45b7-a88c-1238c81586a4 None admin] PATH_INFO: `/v3/users/340decbf23564ee4abce3598f39eb198` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:37.359683 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c540b-2d62-45b7-a88c-1238c81586a4 None admin] RBAC: Authorizing `identity:get_user(user_id=340decbf23564ee4abce3598f39eb198)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:37.361712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c540b-2d62-45b7-a88c-1238c81586a4 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:37.363280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 436/869] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:24:37 2022] GET /identity/v3/users/340decbf23564ee4abce3598f39eb198 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:37.369040 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c896315-392a-4c27-82e2-23540cb25d95 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:37.411191 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:37.465558 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c896315-392a-4c27-82e2-23540cb25d95 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:37.496314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcce59-13e5-4d69-b9c3-7f340f083b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:37.496962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcce59-13e5-4d69-b9c3-7f340f083b29 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:37.497242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcce59-13e5-4d69-b9c3-7f340f083b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:37.497515 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcce59-13e5-4d69-b9c3-7f340f083b29 None admin] PATH_INFO: `/v3/projects/df87e4a656c94a33bb1a0a8d4c9debf5` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:37.498619 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fcce59-13e5-4d69-b9c3-7f340f083b29 None admin] RBAC: Authorizing `identity:get_project(project_id=df87e4a656c94a33bb1a0a8d4c9debf5)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:37.500027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fcce59-13e5-4d69-b9c3-7f340f083b29 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:37.502191 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 434/870] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:24:37 2022] GET /identity/v3/projects/df87e4a656c94a33bb1a0a8d4c9debf5 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:37.508512 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c540b-2d62-45b7-a88c-1238c81586a4 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:37.569084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:37.606944 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c540b-2d62-45b7-a88c-1238c81586a4 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:37.640571 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905acd5-7d2d-4ca8-bd6b-b79089958f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:37.641248 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905acd5-7d2d-4ca8-bd6b-b79089958f95 None admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:37.641528 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905acd5-7d2d-4ca8-bd6b-b79089958f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:37.641801 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905acd5-7d2d-4ca8-bd6b-b79089958f95 None admin] PATH_INFO: `/v3/projects/df87e4a656c94a33bb1a0a8d4c9debf5/users/340decbf23564ee4abce3598f39eb198/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:37.644721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3905acd5-7d2d-4ca8-bd6b-b79089958f95 None admin] RBAC: Authorizing `identity:create_grant(project_id=df87e4a656c94a33bb1a0a8d4c9debf5, user_id=340decbf23564ee4abce3598f39eb198, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:37.646190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3905acd5-7d2d-4ca8-bd6b-b79089958f95 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:37.658412 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 437/871] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:24:37 2022] PUT /identity/v3/projects/df87e4a656c94a33bb1a0a8d4c9debf5/users/340decbf23564ee4abce3598f39eb198/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:24:39.771327 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953f886-3039-47a3-aefe-e652b8e9d382 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:39.771823 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953f886-3039-47a3-aefe-e652b8e9d382 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:39.771823 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953f886-3039-47a3-aefe-e652b8e9d382 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:39.773267 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 435/872] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:24:39.778541 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84c629-a94a-461e-a798-8359a5fc18e3 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:39.778957 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84c629-a94a-461e-a798-8359a5fc18e3 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:39.779089 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd84c629-a94a-461e-a798-8359a5fc18e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:39.797247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-fd84c629-a94a-461e-a798-8359a5fc18e3 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:39.799041 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-fd84c629-a94a-461e-a798-8359a5fc18e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:39.862623 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 438/873] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:24:40.135185 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953f886-3039-47a3-aefe-e652b8e9d382 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:40.239292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953f886-3039-47a3-aefe-e652b8e9d382 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:40.249935 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:40.273637 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c78130-e2af-498b-a1b7-2e24add90804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:40.274251 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c78130-e2af-498b-a1b7-2e24add90804 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:40.274525 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c78130-e2af-498b-a1b7-2e24add90804 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:40.274794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c78130-e2af-498b-a1b7-2e24add90804 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:40.284321 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c78130-e2af-498b-a1b7-2e24add90804 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:24:40.289038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-c1c78130-e2af-498b-a1b7-2e24add90804 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:24:40.289038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:40.289038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:40.289038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:40.289038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:40.289038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:40.289038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:40.289038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:40.289038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:40.289038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:40.289038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:40.289038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:40.289038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:24:40.289038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:24:40.289038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:24:40.289038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:24:40.290105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:24:40.290105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:24:40.290105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:24:40.290105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:24:40.290105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:40.290105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:40.290105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:40.290105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:40.290105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:40.290105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:40.290105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:40.290105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:40.290105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:40.290105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:40.290105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:40.290105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:40.291090 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:40.291090 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:40.291090 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:40.291090 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:40.291090 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:24:40.291090 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:24:40.291090 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:24:40.291090 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:24:40.291090 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:24:40.291090 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:24:40.291090 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:24:40.291090 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:24:40.291090 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 436/874] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 20 13:24:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:24:40.296195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84c629-a94a-461e-a798-8359a5fc18e3 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:40.346971 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:40.393567 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd84c629-a94a-461e-a798-8359a5fc18e3 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:40.426106 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494db284-0955-47e9-8ae0-b8c79572be1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:40.426700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494db284-0955-47e9-8ae0-b8c79572be1b None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:40.426974 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494db284-0955-47e9-8ae0-b8c79572be1b None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:40.427240 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494db284-0955-47e9-8ae0-b8c79572be1b None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:40.427894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494db284-0955-47e9-8ae0-b8c79572be1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:40.428191 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494db284-0955-47e9-8ae0-b8c79572be1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:40.429605 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494db284-0955-47e9-8ae0-b8c79572be1b None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:40.437272 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 439/875] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:24:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:40.443711 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c78130-e2af-498b-a1b7-2e24add90804 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:40.540310 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c78130-e2af-498b-a1b7-2e24add90804 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:40.573595 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8855af-695f-48a8-bbdb-930bd4f3915e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:40.574210 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8855af-695f-48a8-bbdb-930bd4f3915e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:40.574493 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8855af-695f-48a8-bbdb-930bd4f3915e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:40.574767 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8855af-695f-48a8-bbdb-930bd4f3915e None admin] PATH_INFO: `/v3/users/340decbf23564ee4abce3598f39eb198` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:40.575868 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8855af-695f-48a8-bbdb-930bd4f3915e None admin] RBAC: Authorizing `identity:get_user(user_id=340decbf23564ee4abce3598f39eb198)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:40.577313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8855af-695f-48a8-bbdb-930bd4f3915e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:40.578867 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 437/876] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:24:40 2022] GET /identity/v3/users/340decbf23564ee4abce3598f39eb198 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:40.585118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494db284-0955-47e9-8ae0-b8c79572be1b None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:40.683701 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494db284-0955-47e9-8ae0-b8c79572be1b None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:40.696476 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:40.716102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7abab-d53f-4b79-8f9b-bb162572bcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:40.716694 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7abab-d53f-4b79-8f9b-bb162572bcb6 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:40.717011 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7abab-d53f-4b79-8f9b-bb162572bcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:40.717285 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7abab-d53f-4b79-8f9b-bb162572bcb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:40.781287 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb7abab-d53f-4b79-8f9b-bb162572bcb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:40.783511 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb7abab-d53f-4b79-8f9b-bb162572bcb6 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:40.877520 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 440/877] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:24:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 13:24:40.883360 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8855af-695f-48a8-bbdb-930bd4f3915e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:40.921329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:40.977715 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8855af-695f-48a8-bbdb-930bd4f3915e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:41.009889 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef03642-6f06-4ea1-a24e-b7d074c1d3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:41.010490 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef03642-6f06-4ea1-a24e-b7d074c1d3df None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:41.010765 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef03642-6f06-4ea1-a24e-b7d074c1d3df None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:41.011033 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef03642-6f06-4ea1-a24e-b7d074c1d3df None admin] PATH_INFO: `/v3/projects/df87e4a656c94a33bb1a0a8d4c9debf5` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:41.012378 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef03642-6f06-4ea1-a24e-b7d074c1d3df None admin] RBAC: Authorizing `identity:get_project(project_id=df87e4a656c94a33bb1a0a8d4c9debf5)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:41.013813 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef03642-6f06-4ea1-a24e-b7d074c1d3df None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:41.015374 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 438/878] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:24:40 2022] GET /identity/v3/projects/df87e4a656c94a33bb1a0a8d4c9debf5 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:41.021854 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7abab-d53f-4b79-8f9b-bb162572bcb6 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:41.036852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:41.116923 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7abab-d53f-4b79-8f9b-bb162572bcb6 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:41.149831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f459e2-0b39-4735-a516-932da97065b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:41.150427 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f459e2-0b39-4735-a516-932da97065b4 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:41.150710 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f459e2-0b39-4735-a516-932da97065b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:41.150984 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f459e2-0b39-4735-a516-932da97065b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:41.151711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f459e2-0b39-4735-a516-932da97065b4 None admin] RBAC: Adding query filter params (role.id=2bcf35e1bc5f4634b0aa6411fad2f43f, scope.project.id=df87e4a656c94a33bb1a0a8d4c9debf5, user.id=340decbf23564ee4abce3598f39eb198) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:41.152021 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f459e2-0b39-4735-a516-932da97065b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:41.153488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f459e2-0b39-4735-a516-932da97065b4 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:41.159785 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 441/879] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:24:41 2022] GET /identity/v3/role_assignments?user.id=340decbf23564ee4abce3598f39eb198&scope.project.id=df87e4a656c94a33bb1a0a8d4c9debf5&role.id=2bcf35e1bc5f4634b0aa6411fad2f43f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:43.360297 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873590e-4f1e-47d8-a531-5d8760928ea1 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:43.361404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873590e-4f1e-47d8-a531-5d8760928ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:43.361404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873590e-4f1e-47d8-a531-5d8760928ea1 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:43.362716 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 439/880] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:24:43.368033 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8f9a4-44c1-4cce-ab8d-dc139cc7fda8 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:43.368421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8f9a4-44c1-4cce-ab8d-dc139cc7fda8 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:43.368581 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8f9a4-44c1-4cce-ab8d-dc139cc7fda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:43.387807 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-85e8f9a4-44c1-4cce-ab8d-dc139cc7fda8 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:43.388999 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-85e8f9a4-44c1-4cce-ab8d-dc139cc7fda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:43.452289 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 442/881] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:24:43.718202 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873590e-4f1e-47d8-a531-5d8760928ea1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:43.818742 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873590e-4f1e-47d8-a531-5d8760928ea1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:43.825605 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:43.854201 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cac03e-ad8a-49f2-9a88-535c7f3f4c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:43.854794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cac03e-ad8a-49f2-9a88-535c7f3f4c42 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:43.855073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cac03e-ad8a-49f2-9a88-535c7f3f4c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:43.855346 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cac03e-ad8a-49f2-9a88-535c7f3f4c42 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:43.856012 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cac03e-ad8a-49f2-9a88-535c7f3f4c42 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:43.857393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cac03e-ad8a-49f2-9a88-535c7f3f4c42 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:43.872687 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 440/882] 10.222.0.23 () {60 vars in 1138 bytes} [Thu Oct 20 13:24:43 2022] POST /identity/v3/domains => generated 267 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:24:46.029563 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1167d7-862d-410a-9b87-045cf2a12b74 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:46.030116 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1167d7-862d-410a-9b87-045cf2a12b74 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:46.030116 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1167d7-862d-410a-9b87-045cf2a12b74 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:46.031469 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 443/883] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:24:46.036911 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f490ba-8e76-476a-96f4-3b5cc29cdbf6 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:46.037332 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f490ba-8e76-476a-96f4-3b5cc29cdbf6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:46.037462 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f490ba-8e76-476a-96f4-3b5cc29cdbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:46.055500 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-92f490ba-8e76-476a-96f4-3b5cc29cdbf6 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:46.056658 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-92f490ba-8e76-476a-96f4-3b5cc29cdbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:46.119314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 441/884] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:24:46.393437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1167d7-862d-410a-9b87-045cf2a12b74 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:46.404177 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:46.494528 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1167d7-862d-410a-9b87-045cf2a12b74 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:46.530359 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede1fe8-d43b-4940-927c-5ff22e61ccca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:46.530986 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede1fe8-d43b-4940-927c-5ff22e61ccca None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:46.531266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede1fe8-d43b-4940-927c-5ff22e61ccca None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:46.531540 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede1fe8-d43b-4940-927c-5ff22e61ccca None admin] PATH_INFO: `/v3/domains/8e7bdbde1ea342cb9d8a27e9470acecd` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:46.542367 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede1fe8-d43b-4940-927c-5ff22e61ccca None admin] RBAC: Authorizing `identity:get_domain(domain_id=8e7bdbde1ea342cb9d8a27e9470acecd)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:46.543800 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede1fe8-d43b-4940-927c-5ff22e61ccca None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:46.545421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 444/885] 10.222.0.23 () {56 vars in 1183 bytes} [Thu Oct 20 13:24:46 2022] GET /identity/v3/domains/8e7bdbde1ea342cb9d8a27e9470acecd => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:46.551571 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f490ba-8e76-476a-96f4-3b5cc29cdbf6 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:46.638878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:46.645610 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f490ba-8e76-476a-96f4-3b5cc29cdbf6 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:46.677969 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8c4131-00c1-4eee-adb6-ce6126584a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:46.678570 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c4131-00c1-4eee-adb6-ce6126584a16 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:46.678860 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c4131-00c1-4eee-adb6-ce6126584a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:46.679136 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c4131-00c1-4eee-adb6-ce6126584a16 None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:46.679795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c4131-00c1-4eee-adb6-ce6126584a16 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:46.681250 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8c4131-00c1-4eee-adb6-ce6126584a16 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:46.707941 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 442/886] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 20 13:24:46 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:24:48.866212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2cba1-dc7a-43b3-8baf-cacc9322d710 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:48.866749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2cba1-dc7a-43b3-8baf-cacc9322d710 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:48.866749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2cba1-dc7a-43b3-8baf-cacc9322d710 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:48.868085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 445/887] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:24:48.873229 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eed182-193a-4553-a8c9-076d407522f3 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:48.873618 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eed182-193a-4553-a8c9-076d407522f3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:48.873782 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eed182-193a-4553-a8c9-076d407522f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:48.894143 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e4eed182-193a-4553-a8c9-076d407522f3 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:48.895315 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e4eed182-193a-4553-a8c9-076d407522f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:48.960170 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 443/888] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:24:49.229688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f2cba1-dc7a-43b3-8baf-cacc9322d710 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:49.294303 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:49.328185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f2cba1-dc7a-43b3-8baf-cacc9322d710 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:49.360955 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63a060-4aa2-4057-a42a-0c85085794d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:49.361572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63a060-4aa2-4057-a42a-0c85085794d1 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:49.361853 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63a060-4aa2-4057-a42a-0c85085794d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:49.362121 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63a060-4aa2-4057-a42a-0c85085794d1 None admin] PATH_INFO: `/v3/domains/8e7bdbde1ea342cb9d8a27e9470acecd` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:49.363225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63a060-4aa2-4057-a42a-0c85085794d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8e7bdbde1ea342cb9d8a27e9470acecd)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:49.364638 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63a060-4aa2-4057-a42a-0c85085794d1 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:49.366201 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 446/889] 10.222.0.23 () {56 vars in 1183 bytes} [Thu Oct 20 13:24:49 2022] GET /identity/v3/domains/8e7bdbde1ea342cb9d8a27e9470acecd => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:49.371884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eed182-193a-4553-a8c9-076d407522f3 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:49.446968 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:49.468088 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eed182-193a-4553-a8c9-076d407522f3 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:49.501277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43814314-7113-4792-b7e0-759e61f3282c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:49.501885 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43814314-7113-4792-b7e0-759e61f3282c None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:49.502167 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43814314-7113-4792-b7e0-759e61f3282c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:49.502441 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43814314-7113-4792-b7e0-759e61f3282c None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:49.503099 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43814314-7113-4792-b7e0-759e61f3282c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:49.504529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43814314-7113-4792-b7e0-759e61f3282c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:49.525082 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 444/890] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 20 13:24:49 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:24:51.658695 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67086a-d785-4613-bcfe-cda55688773f None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:51.659226 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67086a-d785-4613-bcfe-cda55688773f None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:51.659226 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67086a-d785-4613-bcfe-cda55688773f None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:51.660616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 447/891] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:24:51.665857 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bf314-cd10-4deb-b7bf-b41539268430 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:51.666262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bf314-cd10-4deb-b7bf-b41539268430 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:51.666401 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bf314-cd10-4deb-b7bf-b41539268430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:51.684391 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-424bf314-cd10-4deb-b7bf-b41539268430 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:51.685579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-424bf314-cd10-4deb-b7bf-b41539268430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:51.749919 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 445/892] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:24:52.018768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f67086a-d785-4613-bcfe-cda55688773f None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:52.117336 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f67086a-d785-4613-bcfe-cda55688773f None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:52.149464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7695801-a36a-4ae8-bfdd-d2e592ef2f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:52.150076 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7695801-a36a-4ae8-bfdd-d2e592ef2f1a None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:52.150352 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7695801-a36a-4ae8-bfdd-d2e592ef2f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:52.150623 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7695801-a36a-4ae8-bfdd-d2e592ef2f1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:52.153109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:52.161600 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7695801-a36a-4ae8-bfdd-d2e592ef2f1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:24:52.166314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-f7695801-a36a-4ae8-bfdd-d2e592ef2f1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:24:52.166314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:52.166314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:52.166314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:52.166314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:52.166314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:52.166314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:52.166314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:52.166314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:52.166314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:52.166314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:52.166314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:52.166314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:24:52.166314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:24:52.166314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:24:52.166314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:24:52.167473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:24:52.167473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:24:52.167473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:24:52.167473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:24:52.167473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:52.167473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:52.167473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:52.167473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:52.167473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:52.167473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:52.167473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:52.167473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:52.167473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:52.167473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:52.167473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:52.167473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:52.168551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:52.168551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:52.168551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:52.168551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:52.168551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:24:52.168551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:24:52.168551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:24:52.168551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:24:52.168551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:24:52.168551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:24:52.168551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:24:52.168551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:24:52.168551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 448/893] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 20 13:24:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:24:52.174165 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bf314-cd10-4deb-b7bf-b41539268430 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:52.231927 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:52.269771 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bf314-cd10-4deb-b7bf-b41539268430 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:52.301727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8fb97-ce17-44cb-afed-a931f7ddd80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:52.302315 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8fb97-ce17-44cb-afed-a931f7ddd80e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:52.302593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8fb97-ce17-44cb-afed-a931f7ddd80e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:52.302863 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8fb97-ce17-44cb-afed-a931f7ddd80e None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:52.303519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8fb97-ce17-44cb-afed-a931f7ddd80e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:52.303818 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8fb97-ce17-44cb-afed-a931f7ddd80e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:52.305190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8fb97-ce17-44cb-afed-a931f7ddd80e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:52.312787 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 446/894] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:24:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:52.319203 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7695801-a36a-4ae8-bfdd-d2e592ef2f1a None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:52.415879 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7695801-a36a-4ae8-bfdd-d2e592ef2f1a None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:52.448621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426d455-474f-4d1e-b2b6-a723d12b40e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:52.449264 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426d455-474f-4d1e-b2b6-a723d12b40e9 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:52.449548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426d455-474f-4d1e-b2b6-a723d12b40e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:52.449826 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426d455-474f-4d1e-b2b6-a723d12b40e9 None admin] PATH_INFO: `/v3/users/d1fd9aec6b6b40b88b329b1c6066c2ff` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:52.462389 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426d455-474f-4d1e-b2b6-a723d12b40e9 None admin] RBAC: Authorizing `identity:get_user(user_id=d1fd9aec6b6b40b88b329b1c6066c2ff)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:52.463841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426d455-474f-4d1e-b2b6-a723d12b40e9 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:52.465446 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 449/895] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:24:52 2022] GET /identity/v3/users/d1fd9aec6b6b40b88b329b1c6066c2ff => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:52.471625 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8fb97-ce17-44cb-afed-a931f7ddd80e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:52.569887 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8fb97-ce17-44cb-afed-a931f7ddd80e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:52.593563 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:52.603042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4778e-331d-4782-a759-97a311719162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:52.603665 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4778e-331d-4782-a759-97a311719162 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:52.603950 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4778e-331d-4782-a759-97a311719162 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:52.604226 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4778e-331d-4782-a759-97a311719162 None admin] PATH_INFO: `/v3/projects/281e3822251e4139a263ae1610e9b563` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:52.605353 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf4778e-331d-4782-a759-97a311719162 None admin] RBAC: Authorizing `identity:get_project(project_id=281e3822251e4139a263ae1610e9b563)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:52.606776 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf4778e-331d-4782-a759-97a311719162 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:52.608427 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 447/896] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:24:52 2022] GET /identity/v3/projects/281e3822251e4139a263ae1610e9b563 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:52.615062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426d455-474f-4d1e-b2b6-a723d12b40e9 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:52.668219 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:52.711347 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426d455-474f-4d1e-b2b6-a723d12b40e9 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:52.745141 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28742833-3000-447d-97e5-b7a777ee4617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:52.745791 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28742833-3000-447d-97e5-b7a777ee4617 None admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:52.746071 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28742833-3000-447d-97e5-b7a777ee4617 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:52.746346 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28742833-3000-447d-97e5-b7a777ee4617 None admin] PATH_INFO: `/v3/projects/281e3822251e4139a263ae1610e9b563/users/d1fd9aec6b6b40b88b329b1c6066c2ff/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:52.748209 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28742833-3000-447d-97e5-b7a777ee4617 None admin] RBAC: Authorizing `identity:create_grant(project_id=281e3822251e4139a263ae1610e9b563, user_id=d1fd9aec6b6b40b88b329b1c6066c2ff, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:52.749666 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28742833-3000-447d-97e5-b7a777ee4617 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:52.762787 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 450/897] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:24:52 2022] PUT /identity/v3/projects/281e3822251e4139a263ae1610e9b563/users/d1fd9aec6b6b40b88b329b1c6066c2ff/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:24:54.857110 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704b578-4b41-4d7d-971a-b5e8108298e7 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:54.857665 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704b578-4b41-4d7d-971a-b5e8108298e7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:54.857665 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704b578-4b41-4d7d-971a-b5e8108298e7 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:54.858968 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 448/898] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:24:54.864779 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d0d96-23f7-4391-999d-f3ac24dc4ead None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:54.865215 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d0d96-23f7-4391-999d-f3ac24dc4ead None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:54.865356 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d0d96-23f7-4391-999d-f3ac24dc4ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:54.882949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-ba4d0d96-23f7-4391-999d-f3ac24dc4ead None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:24:54.884111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-ba4d0d96-23f7-4391-999d-f3ac24dc4ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:24:54.947936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 451/899] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:24:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:24:55.222954 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704b578-4b41-4d7d-971a-b5e8108298e7 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:55.320637 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704b578-4b41-4d7d-971a-b5e8108298e7 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:55.353285 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea39e7-3ccf-429b-a9f5-4143462b0185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:55.353896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea39e7-3ccf-429b-a9f5-4143462b0185 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:55.354179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea39e7-3ccf-429b-a9f5-4143462b0185 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:55.354460 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea39e7-3ccf-429b-a9f5-4143462b0185 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:55.365234 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea39e7-3ccf-429b-a9f5-4143462b0185 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:24:55.369977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-70ea39e7-3ccf-429b-a9f5-4143462b0185 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:24:55.369977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:24:55.369977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:24:55.369977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:24:55.369977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:24:55.369977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:24:55.369977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:24:55.369977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:24:55.369977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:24:55.369977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:24:55.369977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:24:55.369977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:24:55.369977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:24:55.369977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:24:55.369977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:24:55.369977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:24:55.371073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:24:55.371073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:24:55.371073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:24:55.371073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:24:55.371073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:24:55.371073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:24:55.371073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:24:55.371073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:24:55.371073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:24:55.371073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:24:55.371073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:24:55.371073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:24:55.371073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:24:55.371073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:24:55.371073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:24:55.371073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:24:55.371900 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:24:55.371900 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:24:55.371900 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:24:55.371900 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:24:55.371900 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:24:55.371900 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:24:55.371900 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:24:55.371900 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:24:55.371900 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:24:55.371900 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:24:55.371900 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:24:55.371900 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:24:55.371900 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 449/900] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 20 13:24:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:24:55.377042 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d0d96-23f7-4391-999d-f3ac24dc4ead None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:55.420846 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:55.474199 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d0d96-23f7-4391-999d-f3ac24dc4ead None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:55.506428 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21a269-7341-405d-ba30-320194dbc4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:55.507024 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21a269-7341-405d-ba30-320194dbc4c1 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:55.507299 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21a269-7341-405d-ba30-320194dbc4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:55.507565 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21a269-7341-405d-ba30-320194dbc4c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:55.508218 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21a269-7341-405d-ba30-320194dbc4c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:55.508514 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21a269-7341-405d-ba30-320194dbc4c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:55.509911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21a269-7341-405d-ba30-320194dbc4c1 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:55.518668 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 452/901] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:24:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:55.524698 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea39e7-3ccf-429b-a9f5-4143462b0185 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:55.577458 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:55.620076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea39e7-3ccf-429b-a9f5-4143462b0185 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:55.653179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46e117-7994-4977-8c48-dbae697262ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:55.653790 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46e117-7994-4977-8c48-dbae697262ab None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:55.654066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46e117-7994-4977-8c48-dbae697262ab None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:55.654334 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46e117-7994-4977-8c48-dbae697262ab None admin] PATH_INFO: `/v3/users/d1fd9aec6b6b40b88b329b1c6066c2ff` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:55.655423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46e117-7994-4977-8c48-dbae697262ab None admin] RBAC: Authorizing `identity:get_user(user_id=d1fd9aec6b6b40b88b329b1c6066c2ff)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:55.656878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46e117-7994-4977-8c48-dbae697262ab None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:55.658409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 450/902] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 20 13:24:55 2022] GET /identity/v3/users/d1fd9aec6b6b40b88b329b1c6066c2ff => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:55.664150 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21a269-7341-405d-ba30-320194dbc4c1 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:55.758820 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21a269-7341-405d-ba30-320194dbc4c1 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:55.765583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:55.792362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2dd2f-5127-4432-aa6c-5564da01f7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:55.792992 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2dd2f-5127-4432-aa6c-5564da01f7e9 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:55.793277 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2dd2f-5127-4432-aa6c-5564da01f7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:55.793547 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2dd2f-5127-4432-aa6c-5564da01f7e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:55.857425 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2dd2f-5127-4432-aa6c-5564da01f7e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:55.858862 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2dd2f-5127-4432-aa6c-5564da01f7e9 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:55.953184 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 453/903] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:24:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 13:24:55.959086 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46e117-7994-4977-8c48-dbae697262ab None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:56.055635 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46e117-7994-4977-8c48-dbae697262ab None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:56.077368 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:56.089007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4774ad9-d48c-44c6-ace9-f370eb68d4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:56.089625 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4774ad9-d48c-44c6-ace9-f370eb68d4bc None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:56.089908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4774ad9-d48c-44c6-ace9-f370eb68d4bc None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:56.090185 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4774ad9-d48c-44c6-ace9-f370eb68d4bc None admin] PATH_INFO: `/v3/projects/281e3822251e4139a263ae1610e9b563` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:56.091291 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4774ad9-d48c-44c6-ace9-f370eb68d4bc None admin] RBAC: Authorizing `identity:get_project(project_id=281e3822251e4139a263ae1610e9b563)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:56.092715 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4774ad9-d48c-44c6-ace9-f370eb68d4bc None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:56.094375 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 451/904] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 20 13:24:55 2022] GET /identity/v3/projects/281e3822251e4139a263ae1610e9b563 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:24:56.100419 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2dd2f-5127-4432-aa6c-5564da01f7e9 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:24:56.108173 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:24:56.196827 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2dd2f-5127-4432-aa6c-5564da01f7e9 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:24:56.229372 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c8167-3cef-446f-b77a-89647804229e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:24:56.229963 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c8167-3cef-446f-b77a-89647804229e None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:24:56.230242 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c8167-3cef-446f-b77a-89647804229e None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:24:56.230514 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c8167-3cef-446f-b77a-89647804229e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:24:56.231228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c8167-3cef-446f-b77a-89647804229e None admin] RBAC: Adding query filter params (role.id=2bcf35e1bc5f4634b0aa6411fad2f43f, scope.project.id=281e3822251e4139a263ae1610e9b563, user.id=d1fd9aec6b6b40b88b329b1c6066c2ff) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:24:56.231533 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c8167-3cef-446f-b77a-89647804229e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:24:56.232998 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c8167-3cef-446f-b77a-89647804229e None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:24:56.239218 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 454/905] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:24:56 2022] GET /identity/v3/role_assignments?user.id=d1fd9aec6b6b40b88b329b1c6066c2ff&scope.project.id=281e3822251e4139a263ae1610e9b563&role.id=2bcf35e1bc5f4634b0aa6411fad2f43f => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:25:06.084991 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:25:06.236649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=65 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:25:52.070497 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa2d61-81e1-4143-8663-92833b236340 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:25:52.070497 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa2d61-81e1-4143-8663-92833b236340 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:25:52.070497 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa2d61-81e1-4143-8663-92833b236340 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:25:52.071809 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 452/906] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:25:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:25:52.078814 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a693b-4196-49e8-b180-35e1e5ffcf2c None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:25:52.079339 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a693b-4196-49e8-b180-35e1e5ffcf2c None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:25:52.079339 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a693b-4196-49e8-b180-35e1e5ffcf2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:25:52.100724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-977a693b-4196-49e8-b180-35e1e5ffcf2c None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:25:52.102220 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-977a693b-4196-49e8-b180-35e1e5ffcf2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:25:52.192892 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 455/907] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:25:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:25:52.468456 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa2d61-81e1-4143-8663-92833b236340 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:25:52.602314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa2d61-81e1-4143-8663-92833b236340 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:25:52.641533 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8a255-d0d2-4526-983b-d105756c3e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:25:52.644640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8a255-d0d2-4526-983b-d105756c3e6b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:25:52.644972 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8a255-d0d2-4526-983b-d105756c3e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:25:52.645255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8a255-d0d2-4526-983b-d105756c3e6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:25:52.658892 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8a255-d0d2-4526-983b-d105756c3e6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:25:52.660980 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8a255-d0d2-4526-983b-d105756c3e6b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:25:52.681621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-d8c8a255-d0d2-4526-983b-d105756c3e6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:25:52.681621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:25:52.681621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:25:52.681621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:25:52.681621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:25:52.681621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:25:52.681621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:25:52.681621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:25:52.681621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:25:52.681621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:25:52.681621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:25:52.681621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:25:52.681621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:25:52.681621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:25:52.681621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:25:52.681621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:25:52.682714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:25:52.682714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:25:52.682714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:25:52.682714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:25:52.682714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:25:52.682714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:25:52.682714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:25:52.682714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:25:52.682714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:25:52.682714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:25:52.682714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:25:52.682714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:25:52.682714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:25:52.682714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:25:52.682714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:25:52.682714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:25:52.682714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:25:52.684199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:25:52.684199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:25:52.684199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:25:52.684199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 453/908] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:25:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:25:52.690559 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a693b-4196-49e8-b180-35e1e5ffcf2c None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:25:52.813659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a693b-4196-49e8-b180-35e1e5ffcf2c None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:25:52.848459 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:25:52.857581 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0169d51-8272-4bbf-a4a7-d017e2cb0be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:25:52.858216 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0169d51-8272-4bbf-a4a7-d017e2cb0be8 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:25:52.858502 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0169d51-8272-4bbf-a4a7-d017e2cb0be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:25:52.858777 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0169d51-8272-4bbf-a4a7-d017e2cb0be8 None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:25:52.859465 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0169d51-8272-4bbf-a4a7-d017e2cb0be8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:25:52.859768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0169d51-8272-4bbf-a4a7-d017e2cb0be8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:25:52.861243 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0169d51-8272-4bbf-a4a7-d017e2cb0be8 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:25:52.879442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 456/909] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:25:52.880284 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8a255-d0d2-4526-983b-d105756c3e6b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:25:52.951732 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:25:53.015149 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8a255-d0d2-4526-983b-d105756c3e6b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:25:53.057197 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b06d6-0701-4f11-8208-ec6fc07386d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:25:53.058002 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b06d6-0701-4f11-8208-ec6fc07386d5 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:25:53.058448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b06d6-0701-4f11-8208-ec6fc07386d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:25:53.058786 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b06d6-0701-4f11-8208-ec6fc07386d5 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:25:53.060112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b06d6-0701-4f11-8208-ec6fc07386d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:25:53.061661 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b06d6-0701-4f11-8208-ec6fc07386d5 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:25:53.085112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 454/910] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 20 13:25:52 2022] POST /identity/v3/users => generated 251 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:25:55.409254 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3df11-bad5-4088-9787-fab6ae19d579 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:25:55.410063 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3df11-bad5-4088-9787-fab6ae19d579 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:25:55.410063 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3df11-bad5-4088-9787-fab6ae19d579 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:25:55.411297 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 457/911] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:25:55.417699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655b13d-a743-4eb4-a33c-5f2b7e24c652 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:25:55.418282 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655b13d-a743-4eb4-a33c-5f2b7e24c652 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:25:55.418411 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655b13d-a743-4eb4-a33c-5f2b7e24c652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:25:55.439834 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-6655b13d-a743-4eb4-a33c-5f2b7e24c652 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:25:55.441118 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6655b13d-a743-4eb4-a33c-5f2b7e24c652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:25:55.516876 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 455/912] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:25:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 13:25:55.819137 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3df11-bad5-4088-9787-fab6ae19d579 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:25:55.935378 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3df11-bad5-4088-9787-fab6ae19d579 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:25:55.974792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8a9e0-e931-46b2-bd92-f9202fcd2202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:25:55.975465 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8a9e0-e931-46b2-bd92-f9202fcd2202 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:25:55.975773 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8a9e0-e931-46b2-bd92-f9202fcd2202 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:25:55.976048 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8a9e0-e931-46b2-bd92-f9202fcd2202 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:25:55.990880 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a8a9e0-e931-46b2-bd92-f9202fcd2202 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:25:55.996369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-83a8a9e0-e931-46b2-bd92-f9202fcd2202 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:25:55.996369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:25:55.996369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:25:55.996369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:25:55.996369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:25:55.996369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:25:55.996369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:25:55.996369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:25:55.996369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:25:55.996369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:25:55.996369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:25:55.996369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:25:55.996369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:25:55.996369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:25:55.996369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:25:55.996369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:25:55.998328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:25:55.998328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:25:55.998328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:25:55.998328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:25:55.998328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:25:55.998328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:25:55.998328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:25:55.998328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:25:55.998328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:25:55.998328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:25:55.998328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:25:55.998328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:25:55.998328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:25:55.998328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:25:55.998328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:25:55.998328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:25:55.999266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:25:55.999266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:25:55.999266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:25:55.999266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:25:55.999266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:25:55.999266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:25:55.999266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:25:55.999266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:25:55.999266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:25:55.999266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:25:55.999266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:25:55.999266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:25:55.999266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 458/913] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 20 13:25:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:25:56.004019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655b13d-a743-4eb4-a33c-5f2b7e24c652 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:25:56.052392 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:25:56.128554 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655b13d-a743-4eb4-a33c-5f2b7e24c652 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:25:56.167040 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76365c59-3e0b-4aa0-ac98-d385dd52fbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:25:56.167603 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76365c59-3e0b-4aa0-ac98-d385dd52fbc2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:25:56.167886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76365c59-3e0b-4aa0-ac98-d385dd52fbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:25:56.168166 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76365c59-3e0b-4aa0-ac98-d385dd52fbc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:25:56.169013 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76365c59-3e0b-4aa0-ac98-d385dd52fbc2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:25:56.169372 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76365c59-3e0b-4aa0-ac98-d385dd52fbc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:25:56.170745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76365c59-3e0b-4aa0-ac98-d385dd52fbc2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:25:56.179284 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 456/914] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 20 13:25:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:25:56.185941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8a9e0-e931-46b2-bd92-f9202fcd2202 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:25:56.242173 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:25:56.302684 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8a9e0-e931-46b2-bd92-f9202fcd2202 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:25:56.345598 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dedaf0-4e5c-46c3-b4c6-24cb6c63c0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:25:56.346296 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dedaf0-4e5c-46c3-b4c6-24cb6c63c0eb None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:25:56.346574 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dedaf0-4e5c-46c3-b4c6-24cb6c63c0eb None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:25:56.346849 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dedaf0-4e5c-46c3-b4c6-24cb6c63c0eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:25:56.358387 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dedaf0-4e5c-46c3-b4c6-24cb6c63c0eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:25:56.359838 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dedaf0-4e5c-46c3-b4c6-24cb6c63c0eb None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:25:56.375098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-63dedaf0-4e5c-46c3-b4c6-24cb6c63c0eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:25:56.375098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:25:56.375098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:25:56.375098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:25:56.375098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:25:56.375098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:25:56.375098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:25:56.375098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:25:56.375098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:25:56.375098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:25:56.375098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:25:56.375098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:25:56.375098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:25:56.375098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:25:56.375098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:25:56.375098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:25:56.376109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:25:56.376109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:25:56.376109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:25:56.376109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:25:56.376109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:25:56.376109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:25:56.376109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:25:56.376109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:25:56.376109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:25:56.376109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:25:56.376109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:25:56.376109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:25:56.376109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:25:56.376109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:25:56.376109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:25:56.376109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:25:56.376109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:25:56.377599 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:25:56.377599 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:25:56.377599 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:25:56.377599 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 459/915] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:25:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:25:56.382362 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76365c59-3e0b-4aa0-ac98-d385dd52fbc2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:25:56.492938 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:25:56.497920 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76365c59-3e0b-4aa0-ac98-d385dd52fbc2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:25:56.540242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f80c1-fd5e-4283-b7bd-7ef79ece410f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:25:56.540886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f80c1-fd5e-4283-b7bd-7ef79ece410f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:25:56.541214 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f80c1-fd5e-4283-b7bd-7ef79ece410f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:25:56.541446 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f80c1-fd5e-4283-b7bd-7ef79ece410f None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:25:56.542076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f80c1-fd5e-4283-b7bd-7ef79ece410f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:25:56.542377 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f80c1-fd5e-4283-b7bd-7ef79ece410f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:25:56.544032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f80c1-fd5e-4283-b7bd-7ef79ece410f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:25:56.562131 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 457/916] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:25:56.563477 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dedaf0-4e5c-46c3-b4c6-24cb6c63c0eb None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:25:56.687466 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dedaf0-4e5c-46c3-b4c6-24cb6c63c0eb None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:25:56.688958 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:25:56.730845 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c484d98-eaef-41e2-a94a-b2a099653446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:25:56.732144 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c484d98-eaef-41e2-a94a-b2a099653446 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:25:56.732531 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c484d98-eaef-41e2-a94a-b2a099653446 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:25:56.732836 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c484d98-eaef-41e2-a94a-b2a099653446 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:25:56.742409 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c484d98-eaef-41e2-a94a-b2a099653446 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:25:56.744387 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c484d98-eaef-41e2-a94a-b2a099653446 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:25:56.756908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-8c484d98-eaef-41e2-a94a-b2a099653446 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 20 13:25:56.756908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:25:56.756908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:25:56.756908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:25:56.756908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:25:56.756908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:25:56.756908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:25:56.756908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:25:56.756908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:25:56.756908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:25:56.756908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:25:56.756908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:25:56.756908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:25:56.756908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:25:56.756908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:25:56.756908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:25:56.758119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:25:56.758119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:25:56.758119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:25:56.758119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:25:56.758119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:25:56.758119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:25:56.758119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:25:56.758119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:25:56.758119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:25:56.758119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:25:56.758119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:25:56.758119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:25:56.758119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:25:56.758119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:25:56.758119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:25:56.758119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:25:56.758119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:25:56.758119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:25:56.759520 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:25:56.759520 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:25:56.759520 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:25:56.759520 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:25:56.759520 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:25:56.759520 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:25:56.759520 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:25:56.759520 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:25:56.759520 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 20 13:25:56.759520 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:25:56.762784 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 460/917] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:25:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:25:56.768571 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f80c1-fd5e-4283-b7bd-7ef79ece410f None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:25:56.876839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f80c1-fd5e-4283-b7bd-7ef79ece410f None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:25:56.918253 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260493e-81d5-4fd4-ba15-cc0a072d8889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:25:56.919188 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260493e-81d5-4fd4-ba15-cc0a072d8889 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:25:56.919628 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260493e-81d5-4fd4-ba15-cc0a072d8889 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:25:56.920014 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260493e-81d5-4fd4-ba15-cc0a072d8889 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:25:56.920847 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260493e-81d5-4fd4-ba15-cc0a072d8889 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:25:56.921272 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260493e-81d5-4fd4-ba15-cc0a072d8889 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:25:56.922797 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260493e-81d5-4fd4-ba15-cc0a072d8889 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:25:56.947288 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 458/918] 10.222.0.23 () {56 vars in 1143 bytes} [Thu Oct 20 13:25:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:25:56.949207 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c484d98-eaef-41e2-a94a-b2a099653446 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:25:57.069541 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c484d98-eaef-41e2-a94a-b2a099653446 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:25:57.110771 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38664c02-77ad-423d-8040-e0bafb719fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:25:57.111866 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38664c02-77ad-423d-8040-e0bafb719fed None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:25:57.112140 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38664c02-77ad-423d-8040-e0bafb719fed None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:25:57.112599 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38664c02-77ad-423d-8040-e0bafb719fed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:25:57.125368 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38664c02-77ad-423d-8040-e0bafb719fed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:25:57.127111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38664c02-77ad-423d-8040-e0bafb719fed None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:25:57.143971 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-38664c02-77ad-423d-8040-e0bafb719fed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:25:57.143971 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:25:57.143971 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:25:57.143971 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:25:57.143971 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:25:57.143971 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:25:57.143971 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:25:57.143971 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:25:57.143971 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:25:57.143971 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:25:57.143971 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:25:57.143971 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:25:57.143971 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:25:57.143971 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:25:57.143971 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:25:57.143971 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:25:57.145214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:25:57.145214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:25:57.145214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:25:57.145214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:25:57.145214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:25:57.145214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:25:57.145214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:25:57.145214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:25:57.145214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:25:57.145214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:25:57.145214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:25:57.145214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:25:57.145214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:25:57.145214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:25:57.145214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:25:57.145214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:25:57.145214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:25:57.146305 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:25:57.146305 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:25:57.146305 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:25:57.150233 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 461/919] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:25:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:25:57.151093 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260493e-81d5-4fd4-ba15-cc0a072d8889 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:25:57.171366 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:25:57.262919 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260493e-81d5-4fd4-ba15-cc0a072d8889 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:25:57.300931 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d677b28-a60b-4207-8755-c1a6f84571ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:25:57.301766 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d677b28-a60b-4207-8755-c1a6f84571ba None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:25:57.302191 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d677b28-a60b-4207-8755-c1a6f84571ba None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:25:57.302646 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d677b28-a60b-4207-8755-c1a6f84571ba None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:25:57.303462 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d677b28-a60b-4207-8755-c1a6f84571ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:25:57.303797 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d677b28-a60b-4207-8755-c1a6f84571ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:25:57.305382 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d677b28-a60b-4207-8755-c1a6f84571ba None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:25:57.322272 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 459/920] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:25:57.323746 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38664c02-77ad-423d-8040-e0bafb719fed None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:25:57.343487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:25:57.446067 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38664c02-77ad-423d-8040-e0bafb719fed None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:25:57.489844 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca30c1e-c651-40d2-b282-95da76a5a62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:25:57.490491 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca30c1e-c651-40d2-b282-95da76a5a62e None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:25:57.490772 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca30c1e-c651-40d2-b282-95da76a5a62e None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:25:57.491045 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca30c1e-c651-40d2-b282-95da76a5a62e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:25:57.501044 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca30c1e-c651-40d2-b282-95da76a5a62e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:25:57.502629 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca30c1e-c651-40d2-b282-95da76a5a62e None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:25:57.520665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-cca30c1e-c651-40d2-b282-95da76a5a62e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:25:57.520665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:25:57.520665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:25:57.520665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:25:57.520665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:25:57.520665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:25:57.520665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:25:57.520665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:25:57.520665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:25:57.520665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:25:57.520665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:25:57.520665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:25:57.520665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:25:57.520665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:25:57.520665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:25:57.526412 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:25:57.526412 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:25:57.526412 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:25:57.526412 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:25:57.526412 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:25:57.526412 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:25:57.526412 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:25:57.526412 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:25:57.526412 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:25:57.526412 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:25:57.526412 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:25:57.526412 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:25:57.526412 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:25:57.526412 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:25:57.526412 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:25:57.526412 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:25:57.526412 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:25:57.527504 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:25:57.527504 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:25:57.527504 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:25:57.527504 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:25:57.527504 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:25:57.527504 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:25:57.527504 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:25:57.527504 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:25:57.527504 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 462/921] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:25:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:25:57.528250 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d677b28-a60b-4207-8755-c1a6f84571ba None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:25:57.621415 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:25:57.650371 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d677b28-a60b-4207-8755-c1a6f84571ba None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:25:57.695384 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07b536-1287-4b60-8dfc-21956c7ae61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:25:57.696105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07b536-1287-4b60-8dfc-21956c7ae61b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:25:57.696388 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07b536-1287-4b60-8dfc-21956c7ae61b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:25:57.696656 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07b536-1287-4b60-8dfc-21956c7ae61b None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:25:57.697421 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07b536-1287-4b60-8dfc-21956c7ae61b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:25:57.697806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07b536-1287-4b60-8dfc-21956c7ae61b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:25:57.699281 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07b536-1287-4b60-8dfc-21956c7ae61b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:25:57.714007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 460/922] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 20 13:25:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:25:57.721167 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca30c1e-c651-40d2-b282-95da76a5a62e None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:25:57.819269 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:25:57.846989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca30c1e-c651-40d2-b282-95da76a5a62e None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:25:57.884270 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b09c2-43e4-4115-9335-901180c98ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:25:57.885052 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b09c2-43e4-4115-9335-901180c98ee2 None admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:25:57.885387 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b09c2-43e4-4115-9335-901180c98ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:25:57.885679 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b09c2-43e4-4115-9335-901180c98ee2 None admin] PATH_INFO: `/v3/projects/4a81b6ecfb6e46a8b08c8bc2b4000fc7/users/2440594934b64621a01c441133778155/roles/0f01a310371a4f6bbae7bb2e319a3b39` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:25:57.902345 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b09c2-43e4-4115-9335-901180c98ee2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a81b6ecfb6e46a8b08c8bc2b4000fc7, user_id=2440594934b64621a01c441133778155, role_id=0f01a310371a4f6bbae7bb2e319a3b39)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:25:57.904069 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b09c2-43e4-4115-9335-901180c98ee2 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:25:57.920571 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 463/923] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:25:57 2022] PUT /identity/v3/projects/4a81b6ecfb6e46a8b08c8bc2b4000fc7/users/2440594934b64621a01c441133778155/roles/0f01a310371a4f6bbae7bb2e319a3b39 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:26:00.194536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976e280-1a81-4555-9c10-961f4a24285a None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:00.195068 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976e280-1a81-4555-9c10-961f4a24285a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:00.195068 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976e280-1a81-4555-9c10-961f4a24285a None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:00.196378 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 461/924] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:26:00.201633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c62a32-8de6-47a9-99ba-e9be462542cf None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:00.202049 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c62a32-8de6-47a9-99ba-e9be462542cf None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:00.202197 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c62a32-8de6-47a9-99ba-e9be462542cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:00.220380 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-08c62a32-8de6-47a9-99ba-e9be462542cf None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:26:00.221611 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-08c62a32-8de6-47a9-99ba-e9be462542cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:26:00.289395 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 464/925] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:26:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:26:00.579348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976e280-1a81-4555-9c10-961f4a24285a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:00.693390 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976e280-1a81-4555-9c10-961f4a24285a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:00.732990 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e86768-9389-423f-a269-dec27fb6d881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:00.733635 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e86768-9389-423f-a269-dec27fb6d881 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:00.733890 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e86768-9389-423f-a269-dec27fb6d881 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:00.734157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e86768-9389-423f-a269-dec27fb6d881 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:00.741102 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:00.745739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e86768-9389-423f-a269-dec27fb6d881 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:26:00.751593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-f1e86768-9389-423f-a269-dec27fb6d881 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:26:00.751593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:26:00.751593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:26:00.751593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:26:00.751593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:26:00.751593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:26:00.751593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:26:00.751593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:26:00.751593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:26:00.751593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:26:00.751593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:26:00.751593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:26:00.751593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:26:00.751593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:26:00.751593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:26:00.751593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:26:00.752804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:26:00.752804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:26:00.752804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:26:00.752804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:26:00.752804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:26:00.752804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:26:00.752804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:26:00.752804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:26:00.752804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:26:00.752804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:26:00.752804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:26:00.752804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:26:00.752804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:26:00.752804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:26:00.752804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:26:00.752804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:26:00.753816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:26:00.753816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:26:00.753816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:26:00.753816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:26:00.753816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:26:00.753816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:26:00.753816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:26:00.753816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:26:00.753816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:26:00.753816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:26:00.753816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 20 13:26:00.753816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:26:00.753816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 462/926] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 20 13:26:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:26:00.759404 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c62a32-8de6-47a9-99ba-e9be462542cf None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:00.817831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:00.871077 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c62a32-8de6-47a9-99ba-e9be462542cf None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:00.910409 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc7354-e1a8-4f3e-9a0a-2cb635c2fb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:00.911147 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc7354-e1a8-4f3e-9a0a-2cb635c2fb6c None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:00.911428 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc7354-e1a8-4f3e-9a0a-2cb635c2fb6c None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:00.911703 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc7354-e1a8-4f3e-9a0a-2cb635c2fb6c None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:00.912471 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc7354-e1a8-4f3e-9a0a-2cb635c2fb6c None admin] RBAC: Adding query filter params (name=service) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:26:00.912757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc7354-e1a8-4f3e-9a0a-2cb635c2fb6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:00.914495 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc7354-e1a8-4f3e-9a0a-2cb635c2fb6c None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:00.923068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 465/927] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 20 13:26:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:26:00.929048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e86768-9389-423f-a269-dec27fb6d881 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:01.044955 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e86768-9389-423f-a269-dec27fb6d881 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:01.083012 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a1bd2-9216-4612-9d58-6ce8f728e40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:01.083654 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a1bd2-9216-4612-9d58-6ce8f728e40b None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:01.083923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a1bd2-9216-4612-9d58-6ce8f728e40b None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:01.084198 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a1bd2-9216-4612-9d58-6ce8f728e40b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:01.096059 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a1bd2-9216-4612-9d58-6ce8f728e40b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:01.097560 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a1bd2-9216-4612-9d58-6ce8f728e40b None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:01.115522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-fa9a1bd2-9216-4612-9d58-6ce8f728e40b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:26:01.115522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:26:01.115522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:26:01.115522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:26:01.115522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:26:01.115522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:26:01.115522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:26:01.115522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:26:01.115522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:26:01.115522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:26:01.115522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:26:01.115522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:26:01.115522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:26:01.115522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:26:01.115522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:26:01.115522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:26:01.116746 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:26:01.116746 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:26:01.116746 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:26:01.116746 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:26:01.116746 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:26:01.116746 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:26:01.116746 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:26:01.116746 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:26:01.116746 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:26:01.116746 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:26:01.116746 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:26:01.116746 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:26:01.116746 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:26:01.116746 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:26:01.116746 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:26:01.116746 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:26:01.116746 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:26:01.117788 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:26:01.117788 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:26:01.117788 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:26:01.117788 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 463/928] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:26:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:26:01.123435 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc7354-e1a8-4f3e-9a0a-2cb635c2fb6c None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:01.227846 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc7354-e1a8-4f3e-9a0a-2cb635c2fb6c None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:01.244649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:01.265969 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01212b77-6e61-439c-8d2f-50d28e4595b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:01.267384 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01212b77-6e61-439c-8d2f-50d28e4595b8 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:01.267384 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01212b77-6e61-439c-8d2f-50d28e4595b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:01.267384 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01212b77-6e61-439c-8d2f-50d28e4595b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:01.267725 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01212b77-6e61-439c-8d2f-50d28e4595b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:26:01.268005 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01212b77-6e61-439c-8d2f-50d28e4595b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:01.269413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01212b77-6e61-439c-8d2f-50d28e4595b8 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:01.281109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 466/929] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:26:01.286857 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a1bd2-9216-4612-9d58-6ce8f728e40b None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:01.341055 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:01.398586 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a1bd2-9216-4612-9d58-6ce8f728e40b None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:01.437626 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb800c3-e1a7-4588-bd3b-4e16cf4abb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:01.438252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb800c3-e1a7-4588-bd3b-4e16cf4abb6c None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:01.438532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb800c3-e1a7-4588-bd3b-4e16cf4abb6c None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:01.438804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb800c3-e1a7-4588-bd3b-4e16cf4abb6c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:01.446291 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb800c3-e1a7-4588-bd3b-4e16cf4abb6c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:01.449086 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb800c3-e1a7-4588-bd3b-4e16cf4abb6c None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:01.460246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-6fb800c3-e1a7-4588-bd3b-4e16cf4abb6c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 20 13:26:01.460246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:26:01.460246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:26:01.460246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:26:01.460246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:26:01.460246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:26:01.460246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:26:01.460246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:26:01.460246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:26:01.460246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:26:01.460246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:26:01.460246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:26:01.460246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:26:01.460246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:26:01.460246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:26:01.460246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:26:01.461541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:26:01.461541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:26:01.461541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:26:01.461541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:26:01.461541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:26:01.461541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:26:01.461541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:26:01.461541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:26:01.461541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:26:01.461541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:26:01.461541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:26:01.461541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:26:01.461541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:26:01.461541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:26:01.461541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:26:01.461541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:26:01.461541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:26:01.461541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:26:01.462719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:26:01.462719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:26:01.462719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:26:01.462719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:26:01.462719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:26:01.462719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:26:01.462719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:26:01.462719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:26:01.462719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 20 13:26:01.462719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:26:01.462719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 464/930] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:26:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:26:01.470421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01212b77-6e61-439c-8d2f-50d28e4595b8 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:01.581274 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01212b77-6e61-439c-8d2f-50d28e4595b8 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:01.618962 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e3c4b-c90a-47c9-9dcf-dbf919736ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:01.619509 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e3c4b-c90a-47c9-9dcf-dbf919736ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:01.619870 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e3c4b-c90a-47c9-9dcf-dbf919736ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:01.620202 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e3c4b-c90a-47c9-9dcf-dbf919736ad1 None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:01.621151 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e3c4b-c90a-47c9-9dcf-dbf919736ad1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:26:01.622113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e3c4b-c90a-47c9-9dcf-dbf919736ad1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:01.623830 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e3c4b-c90a-47c9-9dcf-dbf919736ad1 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:01.644585 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 467/931] 10.222.0.23 () {56 vars in 1143 bytes} [Thu Oct 20 13:26:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:26:01.650174 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb800c3-e1a7-4588-bd3b-4e16cf4abb6c None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:01.760545 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb800c3-e1a7-4588-bd3b-4e16cf4abb6c None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:01.775889 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:01.801262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722e533-fa3d-45b0-8d3d-06684d74002e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:01.801943 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722e533-fa3d-45b0-8d3d-06684d74002e None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:01.802253 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722e533-fa3d-45b0-8d3d-06684d74002e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:01.802562 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722e533-fa3d-45b0-8d3d-06684d74002e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:01.876462 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5722e533-fa3d-45b0-8d3d-06684d74002e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:01.877979 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5722e533-fa3d-45b0-8d3d-06684d74002e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:01.987987 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 465/932] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:26:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 13:26:01.993684 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e3c4b-c90a-47c9-9dcf-dbf919736ad1 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:02.031515 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:02.106333 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e3c4b-c90a-47c9-9dcf-dbf919736ad1 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:02.145907 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc0ccf-ed91-4381-aed4-bb2cea2c9753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:02.146592 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc0ccf-ed91-4381-aed4-bb2cea2c9753 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:02.146867 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc0ccf-ed91-4381-aed4-bb2cea2c9753 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:02.147135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc0ccf-ed91-4381-aed4-bb2cea2c9753 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:02.159350 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc0ccf-ed91-4381-aed4-bb2cea2c9753 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:02.160895 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc0ccf-ed91-4381-aed4-bb2cea2c9753 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:02.177715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-19cc0ccf-ed91-4381-aed4-bb2cea2c9753 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:26:02.177715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:26:02.177715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:26:02.177715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:26:02.177715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:26:02.177715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:26:02.177715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:26:02.177715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:26:02.177715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:26:02.177715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:26:02.177715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:26:02.177715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:26:02.177715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:26:02.177715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:26:02.177715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:26:02.177715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:26:02.179275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:26:02.179275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:26:02.179275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:26:02.179275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:26:02.179275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:26:02.179275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:26:02.179275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:26:02.179275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:26:02.179275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:26:02.179275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:26:02.179275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:26:02.179275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:26:02.179275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:26:02.179275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:26:02.179275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:26:02.179275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:26:02.179275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:26:02.180260 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:26:02.180260 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:26:02.180260 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:26:02.180260 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 468/933] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:26:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:26:02.185519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722e533-fa3d-45b0-8d3d-06684d74002e None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:02.198111 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:02.294996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722e533-fa3d-45b0-8d3d-06684d74002e None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:02.330161 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72892733-ae31-4a0f-97a5-d17c6747e3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:02.330755 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72892733-ae31-4a0f-97a5-d17c6747e3d2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:02.331035 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72892733-ae31-4a0f-97a5-d17c6747e3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:02.331304 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72892733-ae31-4a0f-97a5-d17c6747e3d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:02.331935 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72892733-ae31-4a0f-97a5-d17c6747e3d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:26:02.332241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72892733-ae31-4a0f-97a5-d17c6747e3d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:02.333653 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72892733-ae31-4a0f-97a5-d17c6747e3d2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:02.345974 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 466/934] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:26:02.351714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc0ccf-ed91-4381-aed4-bb2cea2c9753 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:02.443122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:02.461680 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc0ccf-ed91-4381-aed4-bb2cea2c9753 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:02.497024 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84144dbd-2fbf-4993-bb21-e0377c1bd529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:02.497790 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84144dbd-2fbf-4993-bb21-e0377c1bd529 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:02.498124 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84144dbd-2fbf-4993-bb21-e0377c1bd529 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:02.498457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84144dbd-2fbf-4993-bb21-e0377c1bd529 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:02.505735 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84144dbd-2fbf-4993-bb21-e0377c1bd529 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:02.507201 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84144dbd-2fbf-4993-bb21-e0377c1bd529 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:02.521136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-84144dbd-2fbf-4993-bb21-e0377c1bd529 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:26:02.521136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:26:02.521136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:26:02.521136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:26:02.521136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:26:02.521136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:26:02.521136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:26:02.521136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:26:02.521136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:26:02.521136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:26:02.521136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:26:02.521136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:26:02.521136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:26:02.521136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:26:02.521136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:26:02.522125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:26:02.522125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:26:02.522125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:26:02.522125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:26:02.522125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:26:02.522125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:26:02.522125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:26:02.522125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:26:02.522125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:26:02.522125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:26:02.522125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:26:02.522125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:26:02.522125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:26:02.522125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:26:02.522125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:26:02.522125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:26:02.522125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:26:02.523024 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:26:02.523024 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:26:02.523024 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:26:02.523024 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:26:02.523024 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:26:02.523024 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:26:02.523024 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:26:02.523024 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:26:02.523024 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 469/935] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:26:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:26:02.528503 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72892733-ae31-4a0f-97a5-d17c6747e3d2 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:02.603842 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:02.641682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72892733-ae31-4a0f-97a5-d17c6747e3d2 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:02.679249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0b117-cbbf-400a-8ede-07bb307f432f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:02.679855 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0b117-cbbf-400a-8ede-07bb307f432f None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:02.680135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0b117-cbbf-400a-8ede-07bb307f432f None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:02.680413 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0b117-cbbf-400a-8ede-07bb307f432f None admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:02.681143 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0b117-cbbf-400a-8ede-07bb307f432f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:26:02.681496 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0b117-cbbf-400a-8ede-07bb307f432f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:02.683001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0b117-cbbf-400a-8ede-07bb307f432f None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:02.696718 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 467/936] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 20 13:26:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:26:02.703285 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84144dbd-2fbf-4993-bb21-e0377c1bd529 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:02.816731 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84144dbd-2fbf-4993-bb21-e0377c1bd529 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:02.853223 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828cb20b-3043-4152-8765-f020d53ead1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:02.854041 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828cb20b-3043-4152-8765-f020d53ead1f None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:02.854323 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828cb20b-3043-4152-8765-f020d53ead1f None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:02.854672 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828cb20b-3043-4152-8765-f020d53ead1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:02.855573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828cb20b-3043-4152-8765-f020d53ead1f None admin] RBAC: Adding query filter params (role.id=0f01a310371a4f6bbae7bb2e319a3b39, scope.project.id=4a81b6ecfb6e46a8b08c8bc2b4000fc7, user.id=2440594934b64621a01c441133778155) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:26:02.856463 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828cb20b-3043-4152-8765-f020d53ead1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:02.858109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828cb20b-3043-4152-8765-f020d53ead1f None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:02.864701 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 470/937] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:26:02 2022] GET /identity/v3/role_assignments?user.id=2440594934b64621a01c441133778155&scope.project.id=4a81b6ecfb6e46a8b08c8bc2b4000fc7&role.id=0f01a310371a4f6bbae7bb2e319a3b39 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:26:05.104119 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e0d18-c429-4cd5-9b1d-43b661a9bb56 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:05.104720 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e0d18-c429-4cd5-9b1d-43b661a9bb56 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:05.104720 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e0d18-c429-4cd5-9b1d-43b661a9bb56 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:05.106199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 468/938] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:26:05.111678 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3aa49b-2374-402a-b1c1-4a390f716e67 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:05.112094 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3aa49b-2374-402a-b1c1-4a390f716e67 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:05.112214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3aa49b-2374-402a-b1c1-4a390f716e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:05.131606 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-4c3aa49b-2374-402a-b1c1-4a390f716e67 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:26:05.132967 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-4c3aa49b-2374-402a-b1c1-4a390f716e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:26:05.136134 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:05.212661 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 471/939] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:26:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:26:05.496560 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e0d18-c429-4cd5-9b1d-43b661a9bb56 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:05.605840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e0d18-c429-4cd5-9b1d-43b661a9bb56 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:05.634714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:05.641506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bba37-8679-4bd5-b0e3-36177183489a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:05.642140 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bba37-8679-4bd5-b0e3-36177183489a None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:05.642446 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bba37-8679-4bd5-b0e3-36177183489a None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:05.642698 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bba37-8679-4bd5-b0e3-36177183489a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:05.653377 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412bba37-8679-4bd5-b0e3-36177183489a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:26:05.658699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-412bba37-8679-4bd5-b0e3-36177183489a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:26:05.658699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:26:05.658699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:26:05.658699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:26:05.658699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:26:05.658699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:26:05.658699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:26:05.658699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:26:05.658699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:26:05.658699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:26:05.658699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:26:05.658699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:26:05.658699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:26:05.658699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:26:05.658699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:26:05.658699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:26:05.661006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:26:05.661006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:26:05.661006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:26:05.661006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:26:05.661006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:26:05.661006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:26:05.661006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:26:05.661006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:26:05.661006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:26:05.661006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:26:05.661006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:26:05.661006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:26:05.661006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:26:05.661006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:26:05.661006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:26:05.661006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:26:05.661886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:26:05.661886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:26:05.661886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:26:05.661886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:26:05.661886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:26:05.661886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:26:05.661886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:26:05.661886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:26:05.661886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:26:05.661886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:26:05.661886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:26:05.661886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:26:05.661886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 469/940] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 20 13:26:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:26:05.666056 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3aa49b-2374-402a-b1c1-4a390f716e67 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:05.774116 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3aa49b-2374-402a-b1c1-4a390f716e67 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:05.810333 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04e7cd-f478-4a14-b547-f2d2ef53f8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:05.811276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04e7cd-f478-4a14-b547-f2d2ef53f8f0 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:05.811562 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04e7cd-f478-4a14-b547-f2d2ef53f8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:05.811838 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04e7cd-f478-4a14-b547-f2d2ef53f8f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:05.812508 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c04e7cd-f478-4a14-b547-f2d2ef53f8f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:26:05.812817 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c04e7cd-f478-4a14-b547-f2d2ef53f8f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:05.814222 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c04e7cd-f478-4a14-b547-f2d2ef53f8f0 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:05.822350 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 472/941] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:26:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:26:05.828619 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bba37-8679-4bd5-b0e3-36177183489a None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:05.939739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bba37-8679-4bd5-b0e3-36177183489a None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:05.986134 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111bdaa5-0d4e-4892-91ce-03979a8abff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:05.986753 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bdaa5-0d4e-4892-91ce-03979a8abff4 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:05.987059 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bdaa5-0d4e-4892-91ce-03979a8abff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:05.987394 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bdaa5-0d4e-4892-91ce-03979a8abff4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:06.003086 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111bdaa5-0d4e-4892-91ce-03979a8abff4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:06.004658 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111bdaa5-0d4e-4892-91ce-03979a8abff4 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:06.020772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-111bdaa5-0d4e-4892-91ce-03979a8abff4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:26:06.020772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:26:06.020772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:26:06.020772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:26:06.020772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:26:06.020772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:26:06.020772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:26:06.020772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:26:06.020772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:26:06.020772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:26:06.020772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:26:06.020772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:26:06.020772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:26:06.020772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:26:06.020772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:26:06.020772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:26:06.022179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:26:06.022179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:26:06.022179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:26:06.022179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:26:06.022179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:26:06.022179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:26:06.022179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:26:06.022179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:26:06.022179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:26:06.022179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:26:06.022179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:26:06.022179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:26:06.022179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:26:06.022179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:26:06.022179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:26:06.022179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:26:06.022179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:26:06.023404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:26:06.023404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:26:06.023404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:26:06.023404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 470/942] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:26:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:26:06.028783 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04e7cd-f478-4a14-b547-f2d2ef53f8f0 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:06.031983 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:06.151996 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04e7cd-f478-4a14-b547-f2d2ef53f8f0 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:06.192055 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a4a52-e03e-48a1-99a7-7d0542af4226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:06.192788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a4a52-e03e-48a1-99a7-7d0542af4226 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:06.193103 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a4a52-e03e-48a1-99a7-7d0542af4226 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:06.193490 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a4a52-e03e-48a1-99a7-7d0542af4226 None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:06.194180 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a4a52-e03e-48a1-99a7-7d0542af4226 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:26:06.194485 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a4a52-e03e-48a1-99a7-7d0542af4226 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:06.195957 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a4a52-e03e-48a1-99a7-7d0542af4226 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:06.209439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 473/943] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:26:06.216485 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111bdaa5-0d4e-4892-91ce-03979a8abff4 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:06.266694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:06.329601 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111bdaa5-0d4e-4892-91ce-03979a8abff4 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:06.366599 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1831d-2957-42c0-a737-cdcc364a5c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:06.367305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1831d-2957-42c0-a737-cdcc364a5c94 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:06.367622 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1831d-2957-42c0-a737-cdcc364a5c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:06.367891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1831d-2957-42c0-a737-cdcc364a5c94 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:06.376489 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1831d-2957-42c0-a737-cdcc364a5c94 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:06.378146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1831d-2957-42c0-a737-cdcc364a5c94 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:06.389849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-1cd1831d-2957-42c0-a737-cdcc364a5c94 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 20 13:26:06.389849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:26:06.389849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:26:06.389849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:26:06.389849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:26:06.389849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:26:06.389849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:26:06.389849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:26:06.389849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:26:06.389849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:26:06.389849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:26:06.389849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:26:06.389849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:26:06.389849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:26:06.389849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:26:06.389849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:26:06.390992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:26:06.390992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:26:06.390992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:26:06.390992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:26:06.390992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:26:06.390992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:26:06.390992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:26:06.390992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:26:06.390992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:26:06.390992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:26:06.390992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:26:06.390992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:26:06.390992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:26:06.390992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:26:06.390992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:26:06.390992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:26:06.390992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:26:06.390992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:26:06.392248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:26:06.392248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:26:06.392248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:26:06.392248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:26:06.392248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:26:06.392248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:26:06.392248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:26:06.392248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:26:06.392248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 20 13:26:06.392248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:26:06.392248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 471/944] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:26:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:26:06.397554 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a4a52-e03e-48a1-99a7-7d0542af4226 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:06.467029 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:06.513360 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a4a52-e03e-48a1-99a7-7d0542af4226 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:06.553854 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01762116-e9ef-40cb-a6a8-2d360ebd8c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:06.554879 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01762116-e9ef-40cb-a6a8-2d360ebd8c99 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:06.555319 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01762116-e9ef-40cb-a6a8-2d360ebd8c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:06.555589 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01762116-e9ef-40cb-a6a8-2d360ebd8c99 None admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:06.556413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01762116-e9ef-40cb-a6a8-2d360ebd8c99 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:26:06.556724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01762116-e9ef-40cb-a6a8-2d360ebd8c99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:06.558682 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01762116-e9ef-40cb-a6a8-2d360ebd8c99 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:06.578053 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 474/945] 10.222.0.23 () {56 vars in 1143 bytes} [Thu Oct 20 13:26:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:26:06.583896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1831d-2957-42c0-a737-cdcc364a5c94 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:06.692532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1831d-2957-42c0-a737-cdcc364a5c94 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:06.694265 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:06.729903 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca5935-5939-4926-a421-ddc7eb59d407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:06.730711 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca5935-5939-4926-a421-ddc7eb59d407 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:06.731047 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca5935-5939-4926-a421-ddc7eb59d407 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:06.731606 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca5935-5939-4926-a421-ddc7eb59d407 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:06.742144 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca5935-5939-4926-a421-ddc7eb59d407 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:06.743693 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca5935-5939-4926-a421-ddc7eb59d407 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:06.758884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-fcca5935-5939-4926-a421-ddc7eb59d407 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:26:06.758884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:26:06.758884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:26:06.758884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:26:06.758884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:26:06.758884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:26:06.758884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:26:06.758884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:26:06.758884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:26:06.758884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:26:06.758884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:26:06.758884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:26:06.758884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:26:06.758884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:26:06.758884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:26:06.758884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:26:06.759988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:26:06.759988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:26:06.759988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:26:06.759988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:26:06.759988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:26:06.759988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:26:06.759988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:26:06.759988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:26:06.759988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:26:06.759988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:26:06.759988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:26:06.759988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:26:06.759988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:26:06.759988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:26:06.759988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:26:06.759988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:26:06.759988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:26:06.761087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:26:06.761087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:26:06.761087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:26:06.761087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 472/946] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:26:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:26:06.766389 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01762116-e9ef-40cb-a6a8-2d360ebd8c99 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:06.887411 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01762116-e9ef-40cb-a6a8-2d360ebd8c99 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:06.917057 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:06.933458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d4ad7-6f54-48af-a026-12f9bb78e350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:06.934111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d4ad7-6f54-48af-a026-12f9bb78e350 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:06.934394 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d4ad7-6f54-48af-a026-12f9bb78e350 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:06.934669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d4ad7-6f54-48af-a026-12f9bb78e350 None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:06.935320 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d4ad7-6f54-48af-a026-12f9bb78e350 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:26:06.935667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d4ad7-6f54-48af-a026-12f9bb78e350 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:06.937229 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d4ad7-6f54-48af-a026-12f9bb78e350 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:06.950234 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 475/947] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:26:06.956174 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca5935-5939-4926-a421-ddc7eb59d407 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:07.068784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca5935-5939-4926-a421-ddc7eb59d407 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:07.109051 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780fbb0-e579-4837-8389-d541f9acef06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:07.109816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780fbb0-e579-4837-8389-d541f9acef06 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:07.110126 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780fbb0-e579-4837-8389-d541f9acef06 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:07.110398 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780fbb0-e579-4837-8389-d541f9acef06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:07.118791 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780fbb0-e579-4837-8389-d541f9acef06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:07.120740 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780fbb0-e579-4837-8389-d541f9acef06 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:07.133997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-0780fbb0-e579-4837-8389-d541f9acef06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:26:07.133997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:26:07.133997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:26:07.133997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:26:07.133997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:26:07.133997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:26:07.133997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:26:07.133997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:26:07.133997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:26:07.133997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:26:07.133997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:26:07.133997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:26:07.133997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:26:07.133997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:26:07.133997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:26:07.135151 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:26:07.135151 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:26:07.135151 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:26:07.135151 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:26:07.135151 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:26:07.135151 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:26:07.135151 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:26:07.135151 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:26:07.135151 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:26:07.135151 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:26:07.135151 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:26:07.135151 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:26:07.135151 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:26:07.135151 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:26:07.135151 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:26:07.135151 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:26:07.135151 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:26:07.136245 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:26:07.136245 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:26:07.136245 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:26:07.136245 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:26:07.136245 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:26:07.136245 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:26:07.136245 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:26:07.136245 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:26:07.136245 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 473/948] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:26:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:26:07.142218 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d4ad7-6f54-48af-a026-12f9bb78e350 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:07.261892 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d4ad7-6f54-48af-a026-12f9bb78e350 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:07.301524 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bb868-ba11-440c-8f67-33580b52f12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:07.302176 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bb868-ba11-440c-8f67-33580b52f12d None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:07.302456 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bb868-ba11-440c-8f67-33580b52f12d None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:07.302731 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bb868-ba11-440c-8f67-33580b52f12d None admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:07.303428 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75bb868-ba11-440c-8f67-33580b52f12d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:26:07.303738 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75bb868-ba11-440c-8f67-33580b52f12d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:07.305370 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75bb868-ba11-440c-8f67-33580b52f12d None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:07.319964 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 476/949] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 20 13:26:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:26:07.326524 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780fbb0-e579-4837-8389-d541f9acef06 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:07.346123 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:07.447217 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780fbb0-e579-4837-8389-d541f9acef06 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:07.485805 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44056469-44af-4c38-b623-36e34159b82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:07.486464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44056469-44af-4c38-b623-36e34159b82e None admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:07.486750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44056469-44af-4c38-b623-36e34159b82e None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:07.487055 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44056469-44af-4c38-b623-36e34159b82e None admin] PATH_INFO: `/v3/projects/4a81b6ecfb6e46a8b08c8bc2b4000fc7/users/2440594934b64621a01c441133778155/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:07.489176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44056469-44af-4c38-b623-36e34159b82e None admin] RBAC: Authorizing `identity:create_grant(project_id=4a81b6ecfb6e46a8b08c8bc2b4000fc7, user_id=2440594934b64621a01c441133778155, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:07.490801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44056469-44af-4c38-b623-36e34159b82e None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:07.506363 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 474/950] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 20 13:26:07 2022] PUT /identity/v3/projects/4a81b6ecfb6e46a8b08c8bc2b4000fc7/users/2440594934b64621a01c441133778155/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:26:09.913581 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0187fbd-a140-4e31-9cb6-5ae2a5478fef None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:09.914202 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0187fbd-a140-4e31-9cb6-5ae2a5478fef None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:09.914474 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0187fbd-a140-4e31-9cb6-5ae2a5478fef None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:09.916149 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 477/951] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:26:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:26:09.923228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e502ca-87cc-475b-b36b-4f408ff7a20b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:09.923671 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e502ca-87cc-475b-b36b-4f408ff7a20b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:09.923801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e502ca-87cc-475b-b36b-4f408ff7a20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:09.943845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-01e502ca-87cc-475b-b36b-4f408ff7a20b None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:26:09.945199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-01e502ca-87cc-475b-b36b-4f408ff7a20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:26:10.020013 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 475/952] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:26:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:26:10.323795 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0187fbd-a140-4e31-9cb6-5ae2a5478fef None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:10.373498 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:10.435740 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0187fbd-a140-4e31-9cb6-5ae2a5478fef None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:10.472956 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9e79e-5807-47aa-aecb-e4b2c9998634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:10.473586 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9e79e-5807-47aa-aecb-e4b2c9998634 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:10.473864 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9e79e-5807-47aa-aecb-e4b2c9998634 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:10.474133 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9e79e-5807-47aa-aecb-e4b2c9998634 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:10.485236 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9e79e-5807-47aa-aecb-e4b2c9998634 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 20 13:26:10.490342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-9df9e79e-5807-47aa-aecb-e4b2c9998634 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:26:10.490342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:26:10.490342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:26:10.490342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:26:10.490342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:26:10.490342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:26:10.490342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:26:10.490342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:26:10.490342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:26:10.490342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:26:10.490342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:26:10.490342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:26:10.490342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 20 13:26:10.490342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 20 13:26:10.490342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 20 13:26:10.490342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 20 13:26:10.491470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 20 13:26:10.491470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 20 13:26:10.491470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 20 13:26:10.491470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 20 13:26:10.491470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:26:10.491470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:26:10.491470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:26:10.491470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:26:10.491470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:26:10.491470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:26:10.491470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:26:10.491470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:26:10.491470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:26:10.491470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:26:10.491470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:26:10.491470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:26:10.492461 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:26:10.492461 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:26:10.492461 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:26:10.492461 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:26:10.492461 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 20 13:26:10.492461 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 20 13:26:10.492461 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 20 13:26:10.492461 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 20 13:26:10.492461 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 20 13:26:10.492461 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 20 13:26:10.492461 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 20 13:26:10.492461 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:26:10.492461 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 478/953] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 20 13:26:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 20 13:26:10.497997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e502ca-87cc-475b-b36b-4f408ff7a20b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:10.501390 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:10.601596 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e502ca-87cc-475b-b36b-4f408ff7a20b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:10.637214 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d18ee-c093-4d50-afc1-d3b909fa53eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:10.637822 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d18ee-c093-4d50-afc1-d3b909fa53eb None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:10.638111 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d18ee-c093-4d50-afc1-d3b909fa53eb None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:10.638385 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d18ee-c093-4d50-afc1-d3b909fa53eb None admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:10.639054 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d18ee-c093-4d50-afc1-d3b909fa53eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:26:10.639360 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d18ee-c093-4d50-afc1-d3b909fa53eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:10.640732 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d18ee-c093-4d50-afc1-d3b909fa53eb None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:10.648783 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 476/954] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 20 13:26:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:26:10.655600 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9e79e-5807-47aa-aecb-e4b2c9998634 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:10.763614 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:10.766844 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9e79e-5807-47aa-aecb-e4b2c9998634 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:10.802478 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c82d7-38e3-4fc3-b3c9-97454fe7d63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:10.803146 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c82d7-38e3-4fc3-b3c9-97454fe7d63a None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:10.803427 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c82d7-38e3-4fc3-b3c9-97454fe7d63a None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:10.803702 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c82d7-38e3-4fc3-b3c9-97454fe7d63a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:10.814899 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c82d7-38e3-4fc3-b3c9-97454fe7d63a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:10.816362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c82d7-38e3-4fc3-b3c9-97454fe7d63a None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:10.831831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-b34c82d7-38e3-4fc3-b3c9-97454fe7d63a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:26:10.831831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:26:10.831831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:26:10.831831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:26:10.831831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:26:10.831831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:26:10.831831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:26:10.831831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:26:10.831831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:26:10.831831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:26:10.831831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:26:10.831831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:26:10.831831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:26:10.831831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:26:10.831831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:26:10.831831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:26:10.833108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:26:10.833108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:26:10.833108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:26:10.833108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:26:10.833108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:26:10.833108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:26:10.833108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:26:10.833108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:26:10.833108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:26:10.833108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:26:10.833108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:26:10.833108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:26:10.833108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:26:10.833108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:26:10.833108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:26:10.833108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:26:10.833108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:26:10.834256 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:26:10.834256 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:26:10.834256 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:26:10.834256 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 479/955] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:26:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:26:10.838929 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d18ee-c093-4d50-afc1-d3b909fa53eb None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:10.915012 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:10.954423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d18ee-c093-4d50-afc1-d3b909fa53eb None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:10.992943 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6928b90-579f-4d3c-a2e7-827cd1969038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:10.993658 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6928b90-579f-4d3c-a2e7-827cd1969038 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:10.994006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6928b90-579f-4d3c-a2e7-827cd1969038 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:10.994351 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6928b90-579f-4d3c-a2e7-827cd1969038 None admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:10.995097 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6928b90-579f-4d3c-a2e7-827cd1969038 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:26:10.995487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6928b90-579f-4d3c-a2e7-827cd1969038 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:10.996993 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6928b90-579f-4d3c-a2e7-827cd1969038 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:11.009752 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 477/956] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:26:11.016501 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c82d7-38e3-4fc3-b3c9-97454fe7d63a None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:11.137309 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c82d7-38e3-4fc3-b3c9-97454fe7d63a None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:11.180222 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306f684-8af8-413d-9d97-b8a1fee6a9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:11.180790 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306f684-8af8-413d-9d97-b8a1fee6a9a2 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:11.181105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306f684-8af8-413d-9d97-b8a1fee6a9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:11.181379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306f684-8af8-413d-9d97-b8a1fee6a9a2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:11.190474 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1306f684-8af8-413d-9d97-b8a1fee6a9a2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:11.191962 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1306f684-8af8-413d-9d97-b8a1fee6a9a2 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:11.204548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-1306f684-8af8-413d-9d97-b8a1fee6a9a2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 20 13:26:11.204548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:26:11.204548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:26:11.204548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:26:11.204548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:26:11.204548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:26:11.204548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:26:11.204548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:26:11.204548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:26:11.204548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:26:11.204548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:26:11.204548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:26:11.204548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 20 13:26:11.204548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 20 13:26:11.204548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 20 13:26:11.204548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 20 13:26:11.205757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:26:11.205757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:26:11.205757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 20 13:26:11.205757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:26:11.205757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 20 13:26:11.205757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 20 13:26:11.205757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:26:11.205757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:26:11.205757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:26:11.205757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:26:11.205757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:26:11.205757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:26:11.205757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:26:11.205757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:26:11.205757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:26:11.205757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:26:11.205757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:26:11.205757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:26:11.206909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:26:11.206909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:26:11.206909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 20 13:26:11.206909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 20 13:26:11.206909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 20 13:26:11.206909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 20 13:26:11.206909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 20 13:26:11.206909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 20 13:26:11.206909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 20 13:26:11.206909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:26:11.207763 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 480/957] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:26:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:26:11.211883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6928b90-579f-4d3c-a2e7-827cd1969038 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:11.317740 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6928b90-579f-4d3c-a2e7-827cd1969038 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:11.347248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:11.357408 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d29c4f-0c6c-412a-95ea-01feaa654d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:11.358022 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d29c4f-0c6c-412a-95ea-01feaa654d15 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:11.358299 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d29c4f-0c6c-412a-95ea-01feaa654d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:11.358566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d29c4f-0c6c-412a-95ea-01feaa654d15 None admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:11.359337 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d29c4f-0c6c-412a-95ea-01feaa654d15 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:26:11.359641 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d29c4f-0c6c-412a-95ea-01feaa654d15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:11.361100 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d29c4f-0c6c-412a-95ea-01feaa654d15 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:11.379651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 478/958] 10.222.0.23 () {56 vars in 1143 bytes} [Thu Oct 20 13:26:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:26:11.385828 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306f684-8af8-413d-9d97-b8a1fee6a9a2 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:11.401951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:11.491590 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306f684-8af8-413d-9d97-b8a1fee6a9a2 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:11.528387 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b3e5e-8e69-4610-afa1-98d8cb7dadbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:11.529052 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b3e5e-8e69-4610-afa1-98d8cb7dadbf None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:11.529338 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b3e5e-8e69-4610-afa1-98d8cb7dadbf None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:11.529618 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b3e5e-8e69-4610-afa1-98d8cb7dadbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:11.598144 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b3e5e-8e69-4610-afa1-98d8cb7dadbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:11.599602 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b3e5e-8e69-4610-afa1-98d8cb7dadbf None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:11.613634 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:11.699382 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 481/959] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 20 13:26:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 20 13:26:11.705377 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d29c4f-0c6c-412a-95ea-01feaa654d15 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:11.805424 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d29c4f-0c6c-412a-95ea-01feaa654d15 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:11.839777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e4f7c-1a52-4607-a2ac-5ee912f13768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:11.840399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e4f7c-1a52-4607-a2ac-5ee912f13768 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:11.840676 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e4f7c-1a52-4607-a2ac-5ee912f13768 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:11.840973 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e4f7c-1a52-4607-a2ac-5ee912f13768 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:11.850498 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e4f7c-1a52-4607-a2ac-5ee912f13768 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:11.851925 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e4f7c-1a52-4607-a2ac-5ee912f13768 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:11.866532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-f25e4f7c-1a52-4607-a2ac-5ee912f13768 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:26:11.866532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:26:11.866532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:26:11.866532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:26:11.866532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:26:11.866532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:26:11.866532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:26:11.866532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:26:11.866532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:26:11.866532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:26:11.866532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:26:11.866532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:26:11.866532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 20 13:26:11.866532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 20 13:26:11.866532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 20 13:26:11.866532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 20 13:26:11.867587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:26:11.867587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:26:11.867587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:26:11.867587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:26:11.867587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:26:11.867587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:26:11.867587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:26:11.867587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:26:11.867587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:26:11.867587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:26:11.867587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:26:11.867587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:26:11.867587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:26:11.867587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:26:11.867587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:26:11.867587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:26:11.867587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 20 13:26:11.868673 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 20 13:26:11.868673 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 20 13:26:11.868673 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:26:11.868673 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 479/960] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:26:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:26:11.874193 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b3e5e-8e69-4610-afa1-98d8cb7dadbf None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:11.969740 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b3e5e-8e69-4610-afa1-98d8cb7dadbf None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:11.982892 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:12.003605 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddc1bc-2ed3-4a71-a7ef-75c52c5af03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:12.004200 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddc1bc-2ed3-4a71-a7ef-75c52c5af03e None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:12.004482 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddc1bc-2ed3-4a71-a7ef-75c52c5af03e None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:12.004770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddc1bc-2ed3-4a71-a7ef-75c52c5af03e None admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:12.005430 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddc1bc-2ed3-4a71-a7ef-75c52c5af03e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:26:12.005735 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddc1bc-2ed3-4a71-a7ef-75c52c5af03e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:12.007098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddc1bc-2ed3-4a71-a7ef-75c52c5af03e None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:12.018284 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 482/961] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 20 13:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:26:12.023875 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e4f7c-1a52-4607-a2ac-5ee912f13768 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:12.059173 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:12.132939 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e4f7c-1a52-4607-a2ac-5ee912f13768 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:12.166499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb2a6d-8f1c-4d4c-a18b-baf3a1976624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:12.167118 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb2a6d-8f1c-4d4c-a18b-baf3a1976624 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:12.167393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb2a6d-8f1c-4d4c-a18b-baf3a1976624 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:12.168281 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb2a6d-8f1c-4d4c-a18b-baf3a1976624 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:12.174627 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb2a6d-8f1c-4d4c-a18b-baf3a1976624 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:12.176044 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb2a6d-8f1c-4d4c-a18b-baf3a1976624 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:12.187121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application [None req-12eb2a6d-8f1c-4d4c-a18b-baf3a1976624 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:26:12.187121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:26:12.187121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:26:12.187121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:26:12.187121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:26:12.187121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:26:12.187121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:26:12.187121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:26:12.187121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:26:12.187121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:26:12.187121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:26:12.187121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:26:12.187121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 20 13:26:12.187121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 20 13:26:12.187121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 20 13:26:12.188114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 20 13:26:12.188114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:26:12.188114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:26:12.188114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:26:12.188114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:26:12.188114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:26:12.188114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:26:12.188114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:26:12.188114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application with Lock( Oct 20 13:26:12.188114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:26:12.188114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:26:12.188114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:26:12.188114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:26:12.188114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:26:12.188114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:26:12.188114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:26:12.188114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:26:12.189201 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 20 13:26:12.189201 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 20 13:26:12.189201 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 20 13:26:12.189201 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 20 13:26:12.189201 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 20 13:26:12.189201 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 20 13:26:12.189201 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 20 13:26:12.189201 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: ERROR keystone.server.flask.application Oct 20 13:26:12.189201 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 480/962] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 20 13:26:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:26:12.194140 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddc1bc-2ed3-4a71-a7ef-75c52c5af03e None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:12.289910 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddc1bc-2ed3-4a71-a7ef-75c52c5af03e None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:12.321664 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda81505-229b-44e5-a270-d56cd680c059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:12.322197 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda81505-229b-44e5-a270-d56cd680c059 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:12.322475 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda81505-229b-44e5-a270-d56cd680c059 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:12.322745 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda81505-229b-44e5-a270-d56cd680c059 None admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:12.323422 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda81505-229b-44e5-a270-d56cd680c059 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:26:12.323727 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda81505-229b-44e5-a270-d56cd680c059 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:12.325794 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda81505-229b-44e5-a270-d56cd680c059 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:12.337152 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 483/963] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 20 13:26:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:26:12.343791 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb2a6d-8f1c-4d4c-a18b-baf3a1976624 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:12.433310 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:12.439207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb2a6d-8f1c-4d4c-a18b-baf3a1976624 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:12.471391 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566cfe40-7348-4af2-9c3e-fbd448029a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:12.471971 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cfe40-7348-4af2-9c3e-fbd448029a00 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:12.472250 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cfe40-7348-4af2-9c3e-fbd448029a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:12.472519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cfe40-7348-4af2-9c3e-fbd448029a00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:12.473255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566cfe40-7348-4af2-9c3e-fbd448029a00 None admin] RBAC: Adding query filter params (role.id=2bcf35e1bc5f4634b0aa6411fad2f43f, scope.project.id=4a81b6ecfb6e46a8b08c8bc2b4000fc7, user.id=2440594934b64621a01c441133778155) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:26:12.473558 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566cfe40-7348-4af2-9c3e-fbd448029a00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:12.474959 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566cfe40-7348-4af2-9c3e-fbd448029a00 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:12.480431 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 481/964] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 20 13:26:12 2022] GET /identity/v3/role_assignments?user.id=2440594934b64621a01c441133778155&scope.project.id=4a81b6ecfb6e46a8b08c8bc2b4000fc7&role.id=2bcf35e1bc5f4634b0aa6411fad2f43f => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:26:14.618019 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68dfa6-228f-4142-949a-0836d44f9963 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:14.618586 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68dfa6-228f-4142-949a-0836d44f9963 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:14.618586 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68dfa6-228f-4142-949a-0836d44f9963 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:14.619920 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 484/965] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:26:14.625253 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535f9dc-dc1f-4761-bc41-9fb6aac05a17 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:14.625671 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535f9dc-dc1f-4761-bc41-9fb6aac05a17 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:14.625795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535f9dc-dc1f-4761-bc41-9fb6aac05a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:14.644319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f535f9dc-dc1f-4761-bc41-9fb6aac05a17 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:26:14.645517 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f535f9dc-dc1f-4761-bc41-9fb6aac05a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:26:14.709068 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 482/966] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:26:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:26:14.979982 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68dfa6-228f-4142-949a-0836d44f9963 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:15.010176 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:15.078884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68dfa6-228f-4142-949a-0836d44f9963 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:15.115455 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf10617-17f3-40e4-9c8c-245ebd551fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:15.116077 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf10617-17f3-40e4-9c8c-245ebd551fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:15.116352 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf10617-17f3-40e4-9c8c-245ebd551fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:15.116626 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf10617-17f3-40e4-9c8c-245ebd551fc5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:15.117230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf10617-17f3-40e4-9c8c-245ebd551fc5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:15.118592 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf10617-17f3-40e4-9c8c-245ebd551fc5 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:15.128612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-fdf10617-17f3-40e4-9c8c-245ebd551fc5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 13:26:15.128612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:26:15.128612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:26:15.128612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:26:15.128612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:26:15.128612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:26:15.128612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:26:15.128612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:26:15.128612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 13:26:15.128612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:26:15.128612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:26:15.128612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:26:15.128612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:26:15.128612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:26:15.128612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:26:15.128612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:26:15.128612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:26:15.130105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:26:15.130105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:26:15.130105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:26:15.130105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:26:15.130105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:26:15.130105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:26:15.130105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:26:15.130105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:26:15.130105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:26:15.130105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:26:15.130105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:26:15.130105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:26:15.130105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:26:15.130105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:26:15.130105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:26:15.130105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:26:15.131332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:26:15.131332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:26:15.131332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:26:15.131332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:26:15.131332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:26:15.131332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:26:15.131332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:26:15.131332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:26:15.131332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:26:15.131332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:26:15.131332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:26:15.131332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:26:15.131332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 13:26:15.131332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:26:15.131332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 485/967] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Oct 20 13:26:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:26:15.137360 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535f9dc-dc1f-4761-bc41-9fb6aac05a17 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:15.230340 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:15.239860 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535f9dc-dc1f-4761-bc41-9fb6aac05a17 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:15.273674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8502cb1-c76a-467d-aaba-d0c4b90b8592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:15.274270 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8502cb1-c76a-467d-aaba-d0c4b90b8592 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:15.274550 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8502cb1-c76a-467d-aaba-d0c4b90b8592 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:15.274822 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8502cb1-c76a-467d-aaba-d0c4b90b8592 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:15.275445 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8502cb1-c76a-467d-aaba-d0c4b90b8592 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:26:15.275747 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8502cb1-c76a-467d-aaba-d0c4b90b8592 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:15.277522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8502cb1-c76a-467d-aaba-d0c4b90b8592 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:15.282160 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 483/968] 10.222.0.23 () {56 vars in 1116 bytes} [Thu Oct 20 13:26:15 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:26:15.287912 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf10617-17f3-40e4-9c8c-245ebd551fc5 None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:15.379280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:15.385466 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf10617-17f3-40e4-9c8c-245ebd551fc5 None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:15.417853 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c418579-18df-401c-a241-b49e262344b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:15.418474 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c418579-18df-401c-a241-b49e262344b4 None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:15.418717 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c418579-18df-401c-a241-b49e262344b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:15.418984 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c418579-18df-401c-a241-b49e262344b4 None admin] PATH_INFO: `/v3/services` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:15.419597 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c418579-18df-401c-a241-b49e262344b4 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:26:15.419893 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c418579-18df-401c-a241-b49e262344b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:15.421277 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c418579-18df-401c-a241-b49e262344b4 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:15.425524 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 486/969] 10.222.0.23 () {56 vars in 1116 bytes} [Thu Oct 20 13:26:15 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:26:17.572952 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf67b74-5223-4adc-ad5a-f368a07defec None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:17.573449 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf67b74-5223-4adc-ad5a-f368a07defec None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:17.573610 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf67b74-5223-4adc-ad5a-f368a07defec None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:17.574854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 484/970] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:26:17.579896 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82bf46-81ee-4cb1-91a5-36cd63322e92 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:17.580315 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82bf46-81ee-4cb1-91a5-36cd63322e92 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:17.580448 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82bf46-81ee-4cb1-91a5-36cd63322e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:17.598243 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-0b82bf46-81ee-4cb1-91a5-36cd63322e92 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:26:17.599414 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-0b82bf46-81ee-4cb1-91a5-36cd63322e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:26:17.662898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 487/971] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:26:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:26:17.931366 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf67b74-5223-4adc-ad5a-f368a07defec None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:18.029387 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf67b74-5223-4adc-ad5a-f368a07defec None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:18.062198 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185bc154-513d-4665-a8d9-37c125328ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:18.062782 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bc154-513d-4665-a8d9-37c125328ce3 None admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:18.063062 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bc154-513d-4665-a8d9-37c125328ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:18.063334 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185bc154-513d-4665-a8d9-37c125328ce3 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:18.063972 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185bc154-513d-4665-a8d9-37c125328ce3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:18.065359 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185bc154-513d-4665-a8d9-37c125328ce3 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:18.079267 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 485/972] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 20 13:26:17 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:26:20.238517 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ba53a-1347-4105-b62b-aedf9171a984 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:20.239147 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ba53a-1347-4105-b62b-aedf9171a984 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:20.239147 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ba53a-1347-4105-b62b-aedf9171a984 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:20.240905 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 488/973] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:26:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:26:20.247047 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d674f349-4b5c-402d-a40c-5475d5a4eb52 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:20.247468 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d674f349-4b5c-402d-a40c-5475d5a4eb52 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:20.247585 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d674f349-4b5c-402d-a40c-5475d5a4eb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:20.268275 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d674f349-4b5c-402d-a40c-5475d5a4eb52 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:26:20.269668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d674f349-4b5c-402d-a40c-5475d5a4eb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:26:20.280632 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:20.344627 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 486/974] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:26:20 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 20 13:26:20.615072 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ba53a-1347-4105-b62b-aedf9171a984 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:20.703926 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:20.715340 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ba53a-1347-4105-b62b-aedf9171a984 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:20.747427 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3fc0e-b712-4ce0-beb5-6fcd27b1171f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:20.748043 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3fc0e-b712-4ce0-beb5-6fcd27b1171f None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:20.748325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3fc0e-b712-4ce0-beb5-6fcd27b1171f None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:20.748592 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3fc0e-b712-4ce0-beb5-6fcd27b1171f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:20.749191 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e3fc0e-b712-4ce0-beb5-6fcd27b1171f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:20.750552 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e3fc0e-b712-4ce0-beb5-6fcd27b1171f None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:20.759322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-e2e3fc0e-b712-4ce0-beb5-6fcd27b1171f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 13:26:20.759322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:26:20.759322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:26:20.759322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:26:20.759322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:26:20.759322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:26:20.759322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:26:20.759322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:26:20.759322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 13:26:20.759322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:26:20.759322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:26:20.759322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:26:20.759322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:26:20.759322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:26:20.759322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:26:20.759322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:26:20.759322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:26:20.760464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:26:20.760464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:26:20.760464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:26:20.760464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:26:20.760464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:26:20.760464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:26:20.760464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:26:20.760464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:26:20.760464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:26:20.760464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:26:20.760464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:26:20.760464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:26:20.760464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:26:20.760464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:26:20.760464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:26:20.760464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:26:20.761878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:26:20.761878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:26:20.761878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:26:20.761878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:26:20.761878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:26:20.761878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:26:20.761878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:26:20.761878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:26:20.761878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:26:20.761878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:26:20.761878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:26:20.761878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:26:20.761878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 13:26:20.761878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:26:20.761878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 489/975] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Oct 20 13:26:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:26:20.766865 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d674f349-4b5c-402d-a40c-5475d5a4eb52 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:20.861471 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d674f349-4b5c-402d-a40c-5475d5a4eb52 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:20.894187 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f678178-0b31-43e3-ba88-836c01bb4930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:20.894781 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f678178-0b31-43e3-ba88-836c01bb4930 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:20.895062 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f678178-0b31-43e3-ba88-836c01bb4930 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:20.895338 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f678178-0b31-43e3-ba88-836c01bb4930 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:20.895963 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f678178-0b31-43e3-ba88-836c01bb4930 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:26:20.896268 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f678178-0b31-43e3-ba88-836c01bb4930 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:20.897654 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f678178-0b31-43e3-ba88-836c01bb4930 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:20.902906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 487/976] 10.222.0.23 () {56 vars in 1116 bytes} [Thu Oct 20 13:26:20 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:26:20.908730 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3fc0e-b712-4ce0-beb5-6fcd27b1171f None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:21.004003 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3fc0e-b712-4ce0-beb5-6fcd27b1171f None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:21.036208 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18128582-1aab-4c9e-a329-5f09630051ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:21.036733 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18128582-1aab-4c9e-a329-5f09630051ca None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:21.037038 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18128582-1aab-4c9e-a329-5f09630051ca None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:21.037311 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18128582-1aab-4c9e-a329-5f09630051ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:21.037995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18128582-1aab-4c9e-a329-5f09630051ca None admin] RBAC: Adding query filter params (interface=public, service_id=029f0953569f48a8ae0f5e9dd80a190c, region_id=RegionOne) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:26:21.038297 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18128582-1aab-4c9e-a329-5f09630051ca None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:21.039653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18128582-1aab-4c9e-a329-5f09630051ca None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:21.044965 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 490/977] 10.222.0.23 () {56 vars in 1251 bytes} [Thu Oct 20 13:26:20 2022] GET /identity/v3/endpoints?service_id=029f0953569f48a8ae0f5e9dd80a190c&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:26:21.050676 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f678178-0b31-43e3-ba88-836c01bb4930 None admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:21.054341 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:21.147320 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f678178-0b31-43e3-ba88-836c01bb4930 None admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:21.180132 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebfa77-4c21-4920-9c5c-c10236d58884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:21.180717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebfa77-4c21-4920-9c5c-c10236d58884 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:21.181022 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebfa77-4c21-4920-9c5c-c10236d58884 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:21.181294 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebfa77-4c21-4920-9c5c-c10236d58884 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:21.181888 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebfa77-4c21-4920-9c5c-c10236d58884 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:21.183272 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebfa77-4c21-4920-9c5c-c10236d58884 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:21.189921 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 488/978] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Oct 20 13:26:21 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:26:23.294155 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dde717-d7c6-472f-93e1-662512354103 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:23.294702 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dde717-d7c6-472f-93e1-662512354103 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:23.294702 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dde717-d7c6-472f-93e1-662512354103 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:23.296033 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 491/979] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:26:23.301280 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4dd27-5952-4149-a713-b6f870947058 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:23.301703 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4dd27-5952-4149-a713-b6f870947058 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:23.301810 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4dd27-5952-4149-a713-b6f870947058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:23.320071 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-03f4dd27-5952-4149-a713-b6f870947058 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:26:23.321304 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-03f4dd27-5952-4149-a713-b6f870947058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:26:23.379185 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:23.385112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 489/980] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:26:23 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 20 13:26:23.650084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dde717-d7c6-472f-93e1-662512354103 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:23.669488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:23.749090 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dde717-d7c6-472f-93e1-662512354103 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:23.780992 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f70da2-cb86-40a4-b1f5-3c3259fc070c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:23.781612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f70da2-cb86-40a4-b1f5-3c3259fc070c None admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:23.781892 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f70da2-cb86-40a4-b1f5-3c3259fc070c None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:23.782162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f70da2-cb86-40a4-b1f5-3c3259fc070c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:23.782735 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f70da2-cb86-40a4-b1f5-3c3259fc070c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:23.784095 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f70da2-cb86-40a4-b1f5-3c3259fc070c None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:23.793230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application [None req-05f70da2-cb86-40a4-b1f5-3c3259fc070c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 13:26:23.793230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:26:23.793230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 20 13:26:23.793230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 20 13:26:23.793230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 20 13:26:23.793230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 20 13:26:23.793230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 20 13:26:23.793230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:26:23.793230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 13:26:23.793230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:26:23.793230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 20 13:26:23.793230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:26:23.793230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 20 13:26:23.793230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 20 13:26:23.793230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 20 13:26:23.793230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 20 13:26:23.793230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 20 13:26:23.794450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 20 13:26:23.794450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 20 13:26:23.794450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 20 13:26:23.794450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 20 13:26:23.794450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 20 13:26:23.794450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 20 13:26:23.794450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 20 13:26:23.794450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 20 13:26:23.794450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 20 13:26:23.794450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 20 13:26:23.794450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 20 13:26:23.794450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 20 13:26:23.794450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 20 13:26:23.794450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 20 13:26:23.794450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 20 13:26:23.794450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.get_or_create( Oct 20 13:26:23.795289 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 20 13:26:23.795289 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application with Lock( Oct 20 13:26:23.795289 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 20 13:26:23.795289 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self._enter() Oct 20 13:26:23.795289 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 20 13:26:23.795289 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 20 13:26:23.795289 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 20 13:26:23.795289 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application return self.creator() Oct 20 13:26:23.795289 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 20 13:26:23.795289 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application created_value = creator( Oct 20 13:26:23.795289 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 20 13:26:23.795289 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 20 13:26:23.795289 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 20 13:26:23.795289 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: ERROR keystone.server.flask.application Oct 20 13:26:23.795289 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 492/981] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Oct 20 13:26:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 20 13:26:23.800396 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4dd27-5952-4149-a713-b6f870947058 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:23.896072 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4dd27-5952-4149-a713-b6f870947058 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:23.929118 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a13af8-05ba-46cb-9634-d0fc62d284f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:23.929650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a13af8-05ba-46cb-9634-d0fc62d284f2 None admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:23.929931 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a13af8-05ba-46cb-9634-d0fc62d284f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:23.930202 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a13af8-05ba-46cb-9634-d0fc62d284f2 None admin] PATH_INFO: `/v3/services` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:23.930830 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a13af8-05ba-46cb-9634-d0fc62d284f2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:26:23.931133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a13af8-05ba-46cb-9634-d0fc62d284f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:23.932660 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a13af8-05ba-46cb-9634-d0fc62d284f2 None admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:23.937959 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 490/982] 10.222.0.23 () {56 vars in 1116 bytes} [Thu Oct 20 13:26:23 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:26:23.943549 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f70da2-cb86-40a4-b1f5-3c3259fc070c None admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:26:24.023522 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:24.041683 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f70da2-cb86-40a4-b1f5-3c3259fc070c None admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:26:24.074079 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741ec3d-6071-48fc-8e3f-ee69e93afbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:26:24.074711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741ec3d-6071-48fc-8e3f-ee69e93afbf8 None admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:26:24.074996 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741ec3d-6071-48fc-8e3f-ee69e93afbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:26:24.075268 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741ec3d-6071-48fc-8e3f-ee69e93afbf8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:26:24.075921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1741ec3d-6071-48fc-8e3f-ee69e93afbf8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:26:24.077324 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1741ec3d-6071-48fc-8e3f-ee69e93afbf8 None admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:26:24.097183 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 493/983] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:26:23 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:26:33.935466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=69 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:26:34.092840 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:27:28.039527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280b8c4-f6c6-43ac-b315-03e40b2a0b17 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:28.040587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280b8c4-f6c6-43ac-b315-03e40b2a0b17 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:28.041184 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280b8c4-f6c6-43ac-b315-03e40b2a0b17 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:28.049047 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 491/984] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:27:28 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:27:28.050238 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576ea4a-d4a8-4e80-a0b2-78620f078f0d None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:28.050686 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576ea4a-d4a8-4e80-a0b2-78620f078f0d None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:28.050914 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576ea4a-d4a8-4e80-a0b2-78620f078f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:28.142002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-3576ea4a-d4a8-4e80-a0b2-78620f078f0d None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:27:28.144230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-3576ea4a-d4a8-4e80-a0b2-78620f078f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:28.248344 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 494/985] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:27:28.550017 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e31a3-a4dd-43f9-a5ab-33c6d178b56a None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:28.550328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e31a3-a4dd-43f9-a5ab-33c6d178b56a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:28.550578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e31a3-a4dd-43f9-a5ab-33c6d178b56a None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:28.551977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 492/986] 10.222.0.23 () {54 vars in 964 bytes} [Thu Oct 20 13:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:27:28.559008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ad9e1-6f89-49b8-a18c-de0ad649f472 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:28.559578 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ad9e1-6f89-49b8-a18c-de0ad649f472 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:28.559951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ad9e1-6f89-49b8-a18c-de0ad649f472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:28.647036 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-140ad9e1-6f89-49b8-a18c-de0ad649f472 None None] MFA Rules not processed for user `8a101adfc7a94bf1a4777350e9210a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:27:28.649022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-140ad9e1-6f89-49b8-a18c-de0ad649f472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:28.671720 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:27:28.732501 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 495/987] 10.222.0.23 () {58 vars in 1061 bytes} [Thu Oct 20 13:27:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:27:28.742422 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e31a3-a4dd-43f9-a5ab-33c6d178b56a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:27:28.744435 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-634e31a3-a4dd-43f9-a5ab-33c6d178b56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:28.865707 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e31a3-a4dd-43f9-a5ab-33c6d178b56a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:27:28.950383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c6c9e-8bf4-4cef-a85c-fb74374fac70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:27:28.951000 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c6c9e-8bf4-4cef-a85c-fb74374fac70 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:28.951280 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c6c9e-8bf4-4cef-a85c-fb74374fac70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:28.951552 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c6c9e-8bf4-4cef-a85c-fb74374fac70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:28.953528 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cd7c6c9e-8bf4-4cef-a85c-fb74374fac70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:29.022927 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:27:29.095578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c6c9e-8bf4-4cef-a85c-fb74374fac70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:27:29.097953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c6c9e-8bf4-4cef-a85c-fb74374fac70 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:27:29.124084 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 493/988] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:27:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:27:31.661775 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572a6c1-5a93-4890-a938-e0ea2019636c None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:31.662313 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572a6c1-5a93-4890-a938-e0ea2019636c None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:31.662313 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a572a6c1-5a93-4890-a938-e0ea2019636c None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:31.664271 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 496/989] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:27:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:27:31.669482 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bec234-83a2-4a87-972e-a7303edbbfc7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:31.669863 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bec234-83a2-4a87-972e-a7303edbbfc7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:31.670041 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bec234-83a2-4a87-972e-a7303edbbfc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:31.749873 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-01bec234-83a2-4a87-972e-a7303edbbfc7 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:27:31.751092 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-01bec234-83a2-4a87-972e-a7303edbbfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:31.847787 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 494/990] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:27:32.132455 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572a6c1-5a93-4890-a938-e0ea2019636c None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:27:32.134066 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-a572a6c1-5a93-4890-a938-e0ea2019636c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:32.235280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a572a6c1-5a93-4890-a938-e0ea2019636c None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:27:32.303413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:27:32.306355 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c35b7d-2c44-445c-9ff0-ece978d17992 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:27:32.307192 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c35b7d-2c44-445c-9ff0-ece978d17992 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:32.307525 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c35b7d-2c44-445c-9ff0-ece978d17992 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:32.307941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c35b7d-2c44-445c-9ff0-ece978d17992 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:32.309834 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-58c35b7d-2c44-445c-9ff0-ece978d17992 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:32.417179 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c35b7d-2c44-445c-9ff0-ece978d17992 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:27:32.418546 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c35b7d-2c44-445c-9ff0-ece978d17992 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:27:32.436287 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 497/991] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:27:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:27:34.869655 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52d4c2-477b-414f-8651-640b1fdcf75e None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:34.870178 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52d4c2-477b-414f-8651-640b1fdcf75e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:34.870178 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52d4c2-477b-414f-8651-640b1fdcf75e None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:34.871524 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 495/992] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:27:34.876818 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf8290-7508-4474-874f-c1e7068aadcb None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:34.877268 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf8290-7508-4474-874f-c1e7068aadcb None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:34.877409 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf8290-7508-4474-874f-c1e7068aadcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:34.942473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-bdcf8290-7508-4474-874f-c1e7068aadcb None None] MFA Rules not processed for user `b7bc816de9e04a01adb1bc809287ea78`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:27:34.943664 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-bdcf8290-7508-4474-874f-c1e7068aadcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:34.995489 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:27:35.039933 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 498/993] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:27:34 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:27:35.344848 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be043a6c-2585-4bd3-b24d-2c6124770888 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:35.345156 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be043a6c-2585-4bd3-b24d-2c6124770888 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:35.345438 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be043a6c-2585-4bd3-b24d-2c6124770888 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:35.346821 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 496/994] 10.222.0.23 () {54 vars in 964 bytes} [Thu Oct 20 13:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:27:35.353109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d358d-2b22-4959-935d-2cb08b964a65 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:35.353399 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d358d-2b22-4959-935d-2cb08b964a65 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:35.353676 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d358d-2b22-4959-935d-2cb08b964a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:35.429665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-058d358d-2b22-4959-935d-2cb08b964a65 None None] MFA Rules not processed for user `8a101adfc7a94bf1a4777350e9210a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:27:35.430854 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-058d358d-2b22-4959-935d-2cb08b964a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:35.504775 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 499/995] 10.222.0.23 () {58 vars in 1061 bytes} [Thu Oct 20 13:27:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:27:35.515149 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be043a6c-2585-4bd3-b24d-2c6124770888 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:27:35.516735 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-be043a6c-2585-4bd3-b24d-2c6124770888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:35.526585 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:27:35.619941 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be043a6c-2585-4bd3-b24d-2c6124770888 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:27:35.692008 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a31057-2298-4527-983b-deec7cdd619d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:27:35.692607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a31057-2298-4527-983b-deec7cdd619d service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:35.692910 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a31057-2298-4527-983b-deec7cdd619d service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:35.693190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a31057-2298-4527-983b-deec7cdd619d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:35.694903 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-87a31057-2298-4527-983b-deec7cdd619d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:35.755069 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:27:35.801907 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a31057-2298-4527-983b-deec7cdd619d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:27:35.803340 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a31057-2298-4527-983b-deec7cdd619d service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:27:35.822155 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 497/996] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:27:35 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:27:38.883170 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd02a4a-01cb-4a29-aa4b-5c753396558c None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:38.883727 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd02a4a-01cb-4a29-aa4b-5c753396558c None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:38.883727 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd02a4a-01cb-4a29-aa4b-5c753396558c None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:38.885103 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 500/997] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:27:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:27:38.890308 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a266f-ae56-4e06-a584-bb786a9a84ae None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:38.890708 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a266f-ae56-4e06-a584-bb786a9a84ae None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:38.890855 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a266f-ae56-4e06-a584-bb786a9a84ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:38.955834 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5a0a266f-ae56-4e06-a584-bb786a9a84ae None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:27:38.957058 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5a0a266f-ae56-4e06-a584-bb786a9a84ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:39.052415 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 498/998] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:27:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:27:39.339417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd02a4a-01cb-4a29-aa4b-5c753396558c None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:27:39.340979 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-6dd02a4a-01cb-4a29-aa4b-5c753396558c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:39.390798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:27:39.442832 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd02a4a-01cb-4a29-aa4b-5c753396558c None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:27:39.516413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894dc86-0dfa-4dc0-a018-1e9635616455 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:27:39.517044 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894dc86-0dfa-4dc0-a018-1e9635616455 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:39.517318 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894dc86-0dfa-4dc0-a018-1e9635616455 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:39.517583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894dc86-0dfa-4dc0-a018-1e9635616455 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:39.519148 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-9894dc86-0dfa-4dc0-a018-1e9635616455 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:39.611801 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:27:39.625250 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9894dc86-0dfa-4dc0-a018-1e9635616455 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:27:39.626689 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9894dc86-0dfa-4dc0-a018-1e9635616455 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:27:39.645246 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 501/999] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:27:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:27:42.476831 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b41195-6c0a-4c2b-a682-34fe61f358cb None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:42.477438 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b41195-6c0a-4c2b-a682-34fe61f358cb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:42.477438 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b41195-6c0a-4c2b-a682-34fe61f358cb None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:42.478974 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 499/1000] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:27:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:27:42.484710 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f3f0c-27a3-421d-8a06-2e7445a87b75 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:42.485249 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f3f0c-27a3-421d-8a06-2e7445a87b75 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:42.485338 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f3f0c-27a3-421d-8a06-2e7445a87b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:42.559432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-6c1f3f0c-27a3-421d-8a06-2e7445a87b75 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:27:42.560661 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-6c1f3f0c-27a3-421d-8a06-2e7445a87b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:42.667478 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 502/1001] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:27:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:27:42.987256 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b41195-6c0a-4c2b-a682-34fe61f358cb None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:27:42.988840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d7b41195-6c0a-4c2b-a682-34fe61f358cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:43.017865 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:27:43.118329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b41195-6c0a-4c2b-a682-34fe61f358cb None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:27:43.207323 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f965b3-8969-4952-bd16-b6467cecc7de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:27:43.207939 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f965b3-8969-4952-bd16-b6467cecc7de service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:43.208216 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f965b3-8969-4952-bd16-b6467cecc7de service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:43.208489 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f965b3-8969-4952-bd16-b6467cecc7de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:43.210189 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d6f965b3-8969-4952-bd16-b6467cecc7de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:43.288217 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:27:43.328204 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f965b3-8969-4952-bd16-b6467cecc7de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:27:43.329741 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f965b3-8969-4952-bd16-b6467cecc7de service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:27:43.348058 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 500/1002] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:27:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:27:45.595376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9a3fa-942e-45c5-8fff-8f1727e73f77 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:45.595917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9a3fa-942e-45c5-8fff-8f1727e73f77 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:45.595917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9a3fa-942e-45c5-8fff-8f1727e73f77 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:45.597331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 503/1003] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:27:45.603028 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba45c96-c8f2-49cf-84a3-efc10df43048 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:45.603470 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba45c96-c8f2-49cf-84a3-efc10df43048 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:45.603573 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba45c96-c8f2-49cf-84a3-efc10df43048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:45.670719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9ba45c96-c8f2-49cf-84a3-efc10df43048 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:27:45.671895 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9ba45c96-c8f2-49cf-84a3-efc10df43048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:45.768126 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 501/1004] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:27:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:27:46.047858 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9a3fa-942e-45c5-8fff-8f1727e73f77 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:27:46.049509 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-f3e9a3fa-942e-45c5-8fff-8f1727e73f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:46.112352 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:27:46.156194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9a3fa-942e-45c5-8fff-8f1727e73f77 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:27:46.231118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad6014-fe1d-41eb-a252-94f066498c7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:27:46.231727 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad6014-fe1d-41eb-a252-94f066498c7f service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:46.232010 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad6014-fe1d-41eb-a252-94f066498c7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:46.232286 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad6014-fe1d-41eb-a252-94f066498c7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:46.233925 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-52ad6014-fe1d-41eb-a252-94f066498c7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:46.339419 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:27:46.342054 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad6014-fe1d-41eb-a252-94f066498c7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:27:46.344126 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad6014-fe1d-41eb-a252-94f066498c7f service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:27:46.362023 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 504/1005] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:27:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:27:48.570418 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a9741-6a9c-45ed-a0f1-a7c3db61b771 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:48.570898 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a9741-6a9c-45ed-a0f1-a7c3db61b771 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:48.570999 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a9741-6a9c-45ed-a0f1-a7c3db61b771 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:48.572307 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 502/1006] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:27:48.577837 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9daa3-3fb0-4f11-ba6a-b55dde59b24c None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:48.578270 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9daa3-3fb0-4f11-ba6a-b55dde59b24c None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:48.578404 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9daa3-3fb0-4f11-ba6a-b55dde59b24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:48.645179 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-9ad9daa3-3fb0-4f11-ba6a-b55dde59b24c None None] MFA Rules not processed for user `b7bc816de9e04a01adb1bc809287ea78`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:27:48.646913 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-9ad9daa3-3fb0-4f11-ba6a-b55dde59b24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:48.741684 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 505/1007] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:27:48 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:27:49.026039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a9741-6a9c-45ed-a0f1-a7c3db61b771 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:27:49.027567 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ac5a9741-6a9c-45ed-a0f1-a7c3db61b771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:49.059276 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:27:49.133715 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a9741-6a9c-45ed-a0f1-a7c3db61b771 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:27:49.206930 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778c2f3-7504-47a0-9158-b09e8e28446a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:27:49.207529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778c2f3-7504-47a0-9158-b09e8e28446a service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:49.207808 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778c2f3-7504-47a0-9158-b09e8e28446a service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:49.208074 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778c2f3-7504-47a0-9158-b09e8e28446a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:49.210302 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3778c2f3-7504-47a0-9158-b09e8e28446a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:49.285660 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:27:49.322743 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3778c2f3-7504-47a0-9158-b09e8e28446a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:27:49.324162 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3778c2f3-7504-47a0-9158-b09e8e28446a service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:27:49.342448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 503/1008] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:27:49 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:27:51.749160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2618b0-81df-4362-8b1e-42d99a62d1f6 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:51.749681 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2618b0-81df-4362-8b1e-42d99a62d1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:51.749681 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2618b0-81df-4362-8b1e-42d99a62d1f6 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:51.750986 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 506/1009] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:27:51.757082 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57eb5d4-b64c-47c7-a534-ffe1eae90c20 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:51.757476 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57eb5d4-b64c-47c7-a534-ffe1eae90c20 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:51.757636 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57eb5d4-b64c-47c7-a534-ffe1eae90c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:51.822912 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c57eb5d4-b64c-47c7-a534-ffe1eae90c20 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:27:51.824111 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c57eb5d4-b64c-47c7-a534-ffe1eae90c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:51.919490 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 504/1010] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:27:52.201844 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2618b0-81df-4362-8b1e-42d99a62d1f6 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:27:52.203385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-4b2618b0-81df-4362-8b1e-42d99a62d1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:52.270660 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:27:52.304598 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2618b0-81df-4362-8b1e-42d99a62d1f6 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:27:52.375063 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8bbd1-d831-49e4-a9cc-8df7b2dedbf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:27:52.375658 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8bbd1-d831-49e4-a9cc-8df7b2dedbf4 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:52.375936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8bbd1-d831-49e4-a9cc-8df7b2dedbf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:52.376206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8bbd1-d831-49e4-a9cc-8df7b2dedbf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:52.377802 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-92a8bbd1-d831-49e4-a9cc-8df7b2dedbf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:52.487090 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8bbd1-d831-49e4-a9cc-8df7b2dedbf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:27:52.488436 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8bbd1-d831-49e4-a9cc-8df7b2dedbf4 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:27:52.499191 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:27:52.508904 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 507/1011] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:27:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:27:54.700085 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c958e2-fa4d-44e3-84ec-3813b8bce9e2 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:54.700632 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c958e2-fa4d-44e3-84ec-3813b8bce9e2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:54.700632 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c958e2-fa4d-44e3-84ec-3813b8bce9e2 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:54.701986 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 505/1012] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:27:54.707082 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376da45-7840-481e-838f-9df2fcc46b4c None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:54.707456 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376da45-7840-481e-838f-9df2fcc46b4c None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:54.707638 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376da45-7840-481e-838f-9df2fcc46b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:54.773522 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-d376da45-7840-481e-838f-9df2fcc46b4c None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:27:54.774638 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-d376da45-7840-481e-838f-9df2fcc46b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:54.867882 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 508/1013] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:27:55.157261 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c958e2-fa4d-44e3-84ec-3813b8bce9e2 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:27:55.158791 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-97c958e2-fa4d-44e3-84ec-3813b8bce9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:55.200107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:27:55.265804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c958e2-fa4d-44e3-84ec-3813b8bce9e2 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:27:55.338058 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a3bd8-82dc-4ed6-b53b-0d65f65127d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:27:55.338644 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a3bd8-82dc-4ed6-b53b-0d65f65127d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:55.338988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a3bd8-82dc-4ed6-b53b-0d65f65127d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:55.339257 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a3bd8-82dc-4ed6-b53b-0d65f65127d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:55.340836 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8f1a3bd8-82dc-4ed6-b53b-0d65f65127d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:55.422204 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:27:55.448062 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a3bd8-82dc-4ed6-b53b-0d65f65127d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:27:55.449513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a3bd8-82dc-4ed6-b53b-0d65f65127d9 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:27:55.466758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 506/1014] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:27:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:27:57.921949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87b168-7b54-48b9-ab14-fae8b03b75f8 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:57.922507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87b168-7b54-48b9-ab14-fae8b03b75f8 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:57.922507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87b168-7b54-48b9-ab14-fae8b03b75f8 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:57.923854 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 509/1015] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:27:57.929570 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e3361-5463-45b4-a0b3-26c740ce3552 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:57.929999 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e3361-5463-45b4-a0b3-26c740ce3552 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:57.930119 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e3361-5463-45b4-a0b3-26c740ce3552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:57.995556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9e2e3361-5463-45b4-a0b3-26c740ce3552 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:27:57.996745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9e2e3361-5463-45b4-a0b3-26c740ce3552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:58.093784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 507/1016] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:27:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:27:58.377486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87b168-7b54-48b9-ab14-fae8b03b75f8 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:27:58.379025 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-6f87b168-7b54-48b9-ab14-fae8b03b75f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:58.454892 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:27:58.487518 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87b168-7b54-48b9-ab14-fae8b03b75f8 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:27:58.561423 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ddd96-4aa7-48fd-9218-2b6d50c2ae7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:27:58.562020 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ddd96-4aa7-48fd-9218-2b6d50c2ae7e service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:27:58.562294 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ddd96-4aa7-48fd-9218-2b6d50c2ae7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:27:58.562560 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ddd96-4aa7-48fd-9218-2b6d50c2ae7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:27:58.564126 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-7e7ddd96-4aa7-48fd-9218-2b6d50c2ae7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:27:58.668697 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ddd96-4aa7-48fd-9218-2b6d50c2ae7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:27:58.670173 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ddd96-4aa7-48fd-9218-2b6d50c2ae7e service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:27:58.688062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 510/1017] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:27:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:28:02.685647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32c345-e172-434c-8a5d-7dcb365fc58c None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:02.686192 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32c345-e172-434c-8a5d-7dcb365fc58c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:02.686192 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32c345-e172-434c-8a5d-7dcb365fc58c None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:02.687538 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 508/1018] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:28:02.693855 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d139a5-5d6d-4819-be60-f323def80cb3 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:02.694462 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d139a5-5d6d-4819-be60-f323def80cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:02.694739 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d139a5-5d6d-4819-be60-f323def80cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:02.700242 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:02.763337 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-01d139a5-5d6d-4819-be60-f323def80cb3 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:02.764429 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-01d139a5-5d6d-4819-be60-f323def80cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:02.863181 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 511/1019] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:28:03.153169 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb32c345-e172-434c-8a5d-7dcb365fc58c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:28:03.154726 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-eb32c345-e172-434c-8a5d-7dcb365fc58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:03.208257 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:03.263800 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb32c345-e172-434c-8a5d-7dcb365fc58c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:28:03.338442 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166a59e-f923-4be5-8a2c-882fda2179d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:28:03.339045 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166a59e-f923-4be5-8a2c-882fda2179d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:03.339319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166a59e-f923-4be5-8a2c-882fda2179d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:03.339584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166a59e-f923-4be5-8a2c-882fda2179d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:03.341185 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5166a59e-f923-4be5-8a2c-882fda2179d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:03.432362 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:03.449855 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5166a59e-f923-4be5-8a2c-882fda2179d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:28:03.451273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5166a59e-f923-4be5-8a2c-882fda2179d6 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:28:03.470021 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 509/1020] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:28:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:28:06.267271 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e07f19-63e1-480b-b6ab-3e3ce1b5918c None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:06.267798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e07f19-63e1-480b-b6ab-3e3ce1b5918c None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:06.267798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e07f19-63e1-480b-b6ab-3e3ce1b5918c None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:06.269079 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 512/1021] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:28:06.274914 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f412ca6-00e8-445b-bcd8-b7950f582eb8 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:06.275309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f412ca6-00e8-445b-bcd8-b7950f582eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:06.275488 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f412ca6-00e8-445b-bcd8-b7950f582eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:06.341926 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1f412ca6-00e8-445b-bcd8-b7950f582eb8 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:06.343119 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1f412ca6-00e8-445b-bcd8-b7950f582eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:06.438181 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 510/1022] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:28:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:28:06.728948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e07f19-63e1-480b-b6ab-3e3ce1b5918c None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:28:06.730543 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-81e07f19-63e1-480b-b6ab-3e3ce1b5918c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:06.822620 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:06.841578 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e07f19-63e1-480b-b6ab-3e3ce1b5918c None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:28:06.916475 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449c4da-439d-4e44-83e1-ee086ccf1ed5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:28:06.917107 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449c4da-439d-4e44-83e1-ee086ccf1ed5 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:06.917384 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449c4da-439d-4e44-83e1-ee086ccf1ed5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:06.917652 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449c4da-439d-4e44-83e1-ee086ccf1ed5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:06.919219 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-2449c4da-439d-4e44-83e1-ee086ccf1ed5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:07.027554 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449c4da-439d-4e44-83e1-ee086ccf1ed5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:28:07.029101 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449c4da-439d-4e44-83e1-ee086ccf1ed5 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:28:07.048015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 513/1023] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:28:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:28:11.164317 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f83253-0055-4765-b03c-9ccf2ca6db89 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:11.164816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f83253-0055-4765-b03c-9ccf2ca6db89 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:11.164953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f83253-0055-4765-b03c-9ccf2ca6db89 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:11.166243 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 511/1024] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:28:11.171586 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f02388-c6d3-43d9-b7ff-512bfdb87fcd None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:11.171997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f02388-c6d3-43d9-b7ff-512bfdb87fcd None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:11.172131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f02388-c6d3-43d9-b7ff-512bfdb87fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:11.188200 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:11.241056 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-d6f02388-c6d3-43d9-b7ff-512bfdb87fcd None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:11.242268 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-d6f02388-c6d3-43d9-b7ff-512bfdb87fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:11.342494 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 514/1025] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:28:11.627010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f83253-0055-4765-b03c-9ccf2ca6db89 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:28:11.628605 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-09f83253-0055-4765-b03c-9ccf2ca6db89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:11.688629 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:11.735458 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f83253-0055-4765-b03c-9ccf2ca6db89 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:28:11.807947 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e62f9-e756-41fc-9292-7c0108abb3ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:28:11.808545 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e62f9-e756-41fc-9292-7c0108abb3ed service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:11.808821 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e62f9-e756-41fc-9292-7c0108abb3ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:11.809124 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e62f9-e756-41fc-9292-7c0108abb3ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:11.810699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-398e62f9-e756-41fc-9292-7c0108abb3ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:11.914224 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:11.918423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e62f9-e756-41fc-9292-7c0108abb3ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:28:11.919836 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e62f9-e756-41fc-9292-7c0108abb3ed service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:28:11.938136 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 512/1026] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:28:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:28:16.845711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e10c0-f77d-4dcc-b466-e9e1516cbdc9 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:16.846263 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e10c0-f77d-4dcc-b466-e9e1516cbdc9 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:16.846263 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e10c0-f77d-4dcc-b466-e9e1516cbdc9 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:16.848604 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 515/1027] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 20 13:28:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:28:16.854595 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ac0b7-cf32-46bd-ba96-7a96d37e6089 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:16.855034 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ac0b7-cf32-46bd-ba96-7a96d37e6089 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:16.855127 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ac0b7-cf32-46bd-ba96-7a96d37e6089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:16.960356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c40ac0b7-cf32-46bd-ba96-7a96d37e6089 None None] MFA Rules not processed for user `2440594934b64621a01c441133778155`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:16.962093 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c40ac0b7-cf32-46bd-ba96-7a96d37e6089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:17.128278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 513/1028] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 20 13:28:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:28:21.340960 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=59 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:21.502851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b29c6-ec2e-4d7f-90bb-201b31185bcc None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:21.503217 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b29c6-ec2e-4d7f-90bb-201b31185bcc None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:21.504322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b29c6-ec2e-4d7f-90bb-201b31185bcc None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:21.507975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 516/1029] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 20 13:28:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:28:21.514880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99bcf2-e993-4e13-a04e-c8684a6be4e1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:21.515362 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99bcf2-e993-4e13-a04e-c8684a6be4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:21.515441 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99bcf2-e993-4e13-a04e-c8684a6be4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:21.577026 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:21.604720 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-cd99bcf2-e993-4e13-a04e-c8684a6be4e1 None None] MFA Rules not processed for user `2440594934b64621a01c441133778155`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:21.605975 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cd99bcf2-e993-4e13-a04e-c8684a6be4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:21.727770 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 514/1030] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 20 13:28:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:28:22.882704 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd3387-6174-4d56-a95f-8264073a6423 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:22.883257 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd3387-6174-4d56-a95f-8264073a6423 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:22.883257 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd3387-6174-4d56-a95f-8264073a6423 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:22.884590 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 517/1031] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 20 13:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:28:22.889758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e5ae82-6b5c-40da-9f52-ed3222eaf441 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:22.890191 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e5ae82-6b5c-40da-9f52-ed3222eaf441 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:22.890440 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e5ae82-6b5c-40da-9f52-ed3222eaf441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:22.965699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a1e5ae82-6b5c-40da-9f52-ed3222eaf441 None None] MFA Rules not processed for user `2440594934b64621a01c441133778155`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:22.966896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a1e5ae82-6b5c-40da-9f52-ed3222eaf441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:23.008061 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:23.073033 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 515/1032] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 20 13:28:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:28:26.565032 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22a147-3717-4bc6-90ea-3430ae461fe4 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:26.566065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22a147-3717-4bc6-90ea-3430ae461fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:26.566176 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22a147-3717-4bc6-90ea-3430ae461fe4 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:26.567447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 518/1033] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:28:26.573060 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5556e7-c0c8-4929-ab3d-353b3ce78808 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:26.573503 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5556e7-c0c8-4929-ab3d-353b3ce78808 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:26.573582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5556e7-c0c8-4929-ab3d-353b3ce78808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:26.648812 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2d5556e7-c0c8-4929-ab3d-353b3ce78808 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:26.650072 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2d5556e7-c0c8-4929-ab3d-353b3ce78808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:26.760310 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 516/1034] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:28:27.059387 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d7379-7f56-40fb-a2cd-7ca75a9ecff9 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:27.059681 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d7379-7f56-40fb-a2cd-7ca75a9ecff9 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:27.059960 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d7379-7f56-40fb-a2cd-7ca75a9ecff9 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:27.061582 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 519/1035] 10.222.0.23 () {54 vars in 961 bytes} [Thu Oct 20 13:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:28:27.070937 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaebef2-8a15-4967-bd20-a1e5f46164c9 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:27.071421 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaebef2-8a15-4967-bd20-a1e5f46164c9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:27.071849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaebef2-8a15-4967-bd20-a1e5f46164c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:27.090397 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:27.160634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-bfaebef2-8a15-4967-bd20-a1e5f46164c9 None None] MFA Rules not processed for user `ee5c2a3855cb4a668177aa19f337778d`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:27.162146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-bfaebef2-8a15-4967-bd20-a1e5f46164c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:27.291821 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 517/1036] 10.222.0.23 () {58 vars in 1058 bytes} [Thu Oct 20 13:28:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:28:27.299828 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429d7379-7f56-40fb-a2cd-7ca75a9ecff9 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:28:27.301539 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-429d7379-7f56-40fb-a2cd-7ca75a9ecff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:27.460464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429d7379-7f56-40fb-a2cd-7ca75a9ecff9 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:28:27.546970 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:27.588670 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb692a-8bf4-4512-8690-f59e3f70a7e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:28:27.589344 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb692a-8bf4-4512-8690-f59e3f70a7e4 service nova] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:27.589573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb692a-8bf4-4512-8690-f59e3f70a7e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:27.589847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb692a-8bf4-4512-8690-f59e3f70a7e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:27.591522 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-48eb692a-8bf4-4512-8690-f59e3f70a7e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:27.689981 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb692a-8bf4-4512-8690-f59e3f70a7e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:28:27.691879 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb692a-8bf4-4512-8690-f59e3f70a7e4 service nova] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:28:27.712003 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 520/1037] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:28:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:28:30.099757 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962a18b-71ce-4eb9-9d1a-02c572c087e8 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:30.100285 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962a18b-71ce-4eb9-9d1a-02c572c087e8 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:30.100285 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962a18b-71ce-4eb9-9d1a-02c572c087e8 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:30.101929 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 518/1038] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:28:30.107357 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd7c33-d09a-48e6-91b8-1cbfc148befa None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:30.107816 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd7c33-d09a-48e6-91b8-1cbfc148befa None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:30.108146 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd7c33-d09a-48e6-91b8-1cbfc148befa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:30.212624 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-5abd7c33-d09a-48e6-91b8-1cbfc148befa None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:30.213889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-5abd7c33-d09a-48e6-91b8-1cbfc148befa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:30.256723 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:30.320349 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 521/1039] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:28:30.621706 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962a18b-71ce-4eb9-9d1a-02c572c087e8 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:28:30.623865 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1962a18b-71ce-4eb9-9d1a-02c572c087e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:30.696295 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:30.800351 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962a18b-71ce-4eb9-9d1a-02c572c087e8 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:28:30.935682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c3da6-98af-440f-bf48-bdc76a9a11b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:28:30.936291 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c3da6-98af-440f-bf48-bdc76a9a11b6 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:30.936569 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c3da6-98af-440f-bf48-bdc76a9a11b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:30.937152 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c3da6-98af-440f-bf48-bdc76a9a11b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:30.938544 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-589c3da6-98af-440f-bf48-bdc76a9a11b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:30.964623 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:31.042449 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c3da6-98af-440f-bf48-bdc76a9a11b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:28:31.044082 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c3da6-98af-440f-bf48-bdc76a9a11b6 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:28:31.067286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 519/1040] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:28:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:28:33.556836 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993c342-0e6a-4325-868a-d488a44ea7c5 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:33.557361 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993c342-0e6a-4325-868a-d488a44ea7c5 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:33.557464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993c342-0e6a-4325-868a-d488a44ea7c5 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:33.558770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 522/1041] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:28:33.564074 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30016db-56ef-495b-a4b7-5dfa3d366959 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:33.564473 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30016db-56ef-495b-a4b7-5dfa3d366959 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:33.564637 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30016db-56ef-495b-a4b7-5dfa3d366959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:33.636142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d30016db-56ef-495b-a4b7-5dfa3d366959 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:33.637547 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d30016db-56ef-495b-a4b7-5dfa3d366959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:33.739736 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:33.749036 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 520/1042] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:28:34.031920 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cb877-63ac-4314-aebc-fad4884d2454 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:34.032586 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cb877-63ac-4314-aebc-fad4884d2454 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:34.033049 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cb877-63ac-4314-aebc-fad4884d2454 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:34.035369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 523/1043] 10.222.0.23 () {54 vars in 961 bytes} [Thu Oct 20 13:28:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:28:34.041092 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dba242-efa8-48b4-bba0-e5db04721b08 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:34.041488 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dba242-efa8-48b4-bba0-e5db04721b08 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:34.041864 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dba242-efa8-48b4-bba0-e5db04721b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:34.121981 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-52dba242-efa8-48b4-bba0-e5db04721b08 None None] MFA Rules not processed for user `ee5c2a3855cb4a668177aa19f337778d`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:34.123342 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-52dba242-efa8-48b4-bba0-e5db04721b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:34.246473 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 521/1044] 10.222.0.23 () {58 vars in 1058 bytes} [Thu Oct 20 13:28:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:28:34.255486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cb877-63ac-4314-aebc-fad4884d2454 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:28:34.257395 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-375cb877-63ac-4314-aebc-fad4884d2454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:34.416324 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cb877-63ac-4314-aebc-fad4884d2454 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:28:34.438657 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:34.547939 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdefff6-5a55-4d0c-a95c-7854dbd81a69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:28:34.548542 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdefff6-5a55-4d0c-a95c-7854dbd81a69 service nova] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:34.548816 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdefff6-5a55-4d0c-a95c-7854dbd81a69 service nova] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:34.549128 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdefff6-5a55-4d0c-a95c-7854dbd81a69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:34.551425 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-dfdefff6-5a55-4d0c-a95c-7854dbd81a69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:34.649602 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdefff6-5a55-4d0c-a95c-7854dbd81a69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:28:34.651219 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdefff6-5a55-4d0c-a95c-7854dbd81a69 service nova] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:28:34.672093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 524/1045] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:28:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:28:34.777636 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f047262-592d-4cc8-ba37-39eef46f360e None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:34.778191 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f047262-592d-4cc8-ba37-39eef46f360e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:34.778191 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f047262-592d-4cc8-ba37-39eef46f360e None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:34.783462 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 522/1046] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:28:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:28:34.785620 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144171b-9c58-4db6-95bc-ce9b76f88c87 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:34.785915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144171b-9c58-4db6-95bc-ce9b76f88c87 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:34.786218 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144171b-9c58-4db6-95bc-ce9b76f88c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:34.829152 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:34.874671 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-e144171b-9c58-4db6-95bc-ce9b76f88c87 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:34.876097 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-e144171b-9c58-4db6-95bc-ce9b76f88c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:35.008308 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 525/1047] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:28:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:28:35.338244 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b91d27-d076-40e4-b9cb-3d7752beb56d None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:35.338244 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b91d27-d076-40e4-b9cb-3d7752beb56d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:35.338244 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b91d27-d076-40e4-b9cb-3d7752beb56d None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:35.341208 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 523/1048] 10.222.0.23 () {54 vars in 963 bytes} [Thu Oct 20 13:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:28:35.344379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9a803-170c-4827-8ff2-1dcc70b7b908 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:35.344662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9a803-170c-4827-8ff2-1dcc70b7b908 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:35.344959 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9a803-170c-4827-8ff2-1dcc70b7b908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:35.425529 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-13b9a803-170c-4827-8ff2-1dcc70b7b908 None None] MFA Rules not processed for user `349ba417c3fc49ecaa11a65066dc54f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:35.426730 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-13b9a803-170c-4827-8ff2-1dcc70b7b908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:35.484570 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:35.521823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 526/1049] 10.222.0.23 () {58 vars in 1060 bytes} [Thu Oct 20 13:28:35 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:28:35.529243 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b91d27-d076-40e4-b9cb-3d7752beb56d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:28:35.530736 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-71b91d27-d076-40e4-b9cb-3d7752beb56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:35.584821 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:35.641604 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b91d27-d076-40e4-b9cb-3d7752beb56d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:28:35.726160 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d0072-63e7-4a9a-8aa4-a0cc684dbe7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:28:35.726971 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d0072-63e7-4a9a-8aa4-a0cc684dbe7b service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:35.727246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d0072-63e7-4a9a-8aa4-a0cc684dbe7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:35.727512 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d0072-63e7-4a9a-8aa4-a0cc684dbe7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:35.729370 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-646d0072-63e7-4a9a-8aa4-a0cc684dbe7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:35.847608 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:35.865506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d0072-63e7-4a9a-8aa4-a0cc684dbe7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:28:35.867269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d0072-63e7-4a9a-8aa4-a0cc684dbe7b service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:28:35.891128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 524/1050] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:28:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:28:36.977065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee9082-69ab-4515-b0ea-2eb44af776ec None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:36.977610 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee9082-69ab-4515-b0ea-2eb44af776ec None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:36.977610 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee9082-69ab-4515-b0ea-2eb44af776ec None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:36.978957 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 527/1051] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:28:36.984175 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7bfcba-2332-4e50-92dd-c908646c47ce None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:36.984583 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7bfcba-2332-4e50-92dd-c908646c47ce None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:36.984725 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7bfcba-2332-4e50-92dd-c908646c47ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:37.051297 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1b7bfcba-2332-4e50-92dd-c908646c47ce None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:37.052495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1b7bfcba-2332-4e50-92dd-c908646c47ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:37.150918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 525/1052] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:28:37.455244 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee9082-69ab-4515-b0ea-2eb44af776ec None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:28:37.456991 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b2ee9082-69ab-4515-b0ea-2eb44af776ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:37.617224 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee9082-69ab-4515-b0ea-2eb44af776ec None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:28:37.670151 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:37.735431 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6a6ca-57f4-4225-a3be-fe698a2bb039 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:28:37.736059 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6a6ca-57f4-4225-a3be-fe698a2bb039 service nova] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:37.736328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6a6ca-57f4-4225-a3be-fe698a2bb039 service nova] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:37.736645 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6a6ca-57f4-4225-a3be-fe698a2bb039 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:37.738386 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-2cf6a6ca-57f4-4225-a3be-fe698a2bb039 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:37.832304 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6a6ca-57f4-4225-a3be-fe698a2bb039 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:28:37.833942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6a6ca-57f4-4225-a3be-fe698a2bb039 service nova] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:28:37.852903 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 528/1053] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:28:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:28:40.062616 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73515b9-a4e3-47b4-8e25-090dfce117f5 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:40.063132 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73515b9-a4e3-47b4-8e25-090dfce117f5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:40.063245 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73515b9-a4e3-47b4-8e25-090dfce117f5 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:40.065252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 526/1054] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:28:40.070245 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84496bf7-4f29-4bdd-96f3-77445fbbfbf0 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:40.070659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84496bf7-4f29-4bdd-96f3-77445fbbfbf0 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:40.070829 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84496bf7-4f29-4bdd-96f3-77445fbbfbf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:40.140021 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-84496bf7-4f29-4bdd-96f3-77445fbbfbf0 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:40.141255 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-84496bf7-4f29-4bdd-96f3-77445fbbfbf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:40.148592 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:40.254947 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 529/1055] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:28:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:28:40.543710 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73515b9-a4e3-47b4-8e25-090dfce117f5 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:28:40.545480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f73515b9-a4e3-47b4-8e25-090dfce117f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:40.612621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:40.702853 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73515b9-a4e3-47b4-8e25-090dfce117f5 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:28:40.822810 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914679d0-eb09-47e2-8134-b137afd91d88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:28:40.823568 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914679d0-eb09-47e2-8134-b137afd91d88 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:40.823969 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914679d0-eb09-47e2-8134-b137afd91d88 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:40.824346 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914679d0-eb09-47e2-8134-b137afd91d88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:40.826065 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-914679d0-eb09-47e2-8134-b137afd91d88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:40.846442 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:40.921320 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914679d0-eb09-47e2-8134-b137afd91d88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:28:40.923049 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914679d0-eb09-47e2-8134-b137afd91d88 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:28:40.942484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 527/1056] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:28:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:28:43.162961 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb8b94-1ab7-4ac7-a272-d61fcf11e517 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:43.163538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb8b94-1ab7-4ac7-a272-d61fcf11e517 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:43.163538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb8b94-1ab7-4ac7-a272-d61fcf11e517 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:43.165036 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 530/1057] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:28:43.170004 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c8ffc-8a36-49e7-96b5-424dd9bbd7cc None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:43.170435 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c8ffc-8a36-49e7-96b5-424dd9bbd7cc None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:43.170556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c8ffc-8a36-49e7-96b5-424dd9bbd7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:43.244060 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-fe8c8ffc-8a36-49e7-96b5-424dd9bbd7cc None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:43.245666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fe8c8ffc-8a36-49e7-96b5-424dd9bbd7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:43.353851 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:43.369259 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 528/1058] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:28:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:28:43.663763 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb8b94-1ab7-4ac7-a272-d61fcf11e517 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:28:43.665241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-afcb8b94-1ab7-4ac7-a272-d61fcf11e517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:43.802136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:43.812937 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb8b94-1ab7-4ac7-a272-d61fcf11e517 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:28:43.927693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75684f-a470-4090-bf3a-3324939e2569 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:28:43.928232 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75684f-a470-4090-bf3a-3324939e2569 service nova] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:43.928561 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75684f-a470-4090-bf3a-3324939e2569 service nova] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:43.928776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75684f-a470-4090-bf3a-3324939e2569 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:43.930776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-4c75684f-a470-4090-bf3a-3324939e2569 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:44.028433 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c75684f-a470-4090-bf3a-3324939e2569 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:28:44.030135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c75684f-a470-4090-bf3a-3324939e2569 service nova] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:28:44.041571 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:44.051467 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 531/1059] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:28:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:28:46.310536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909fc75-fbf2-4ef5-bc79-b8f1566b0ccc None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:46.311067 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909fc75-fbf2-4ef5-bc79-b8f1566b0ccc None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:46.311067 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909fc75-fbf2-4ef5-bc79-b8f1566b0ccc None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:46.312365 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 529/1060] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:28:46.319899 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f33ce-34bc-47c3-bd7e-b1988198921f None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:46.320345 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f33ce-34bc-47c3-bd7e-b1988198921f None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:46.320676 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f33ce-34bc-47c3-bd7e-b1988198921f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:46.406249 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-cd6f33ce-34bc-47c3-bd7e-b1988198921f None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:46.407449 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-cd6f33ce-34bc-47c3-bd7e-b1988198921f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:46.518800 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 532/1061] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:28:46.881195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909fc75-fbf2-4ef5-bc79-b8f1566b0ccc None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:28:46.882739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a909fc75-fbf2-4ef5-bc79-b8f1566b0ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:47.032202 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909fc75-fbf2-4ef5-bc79-b8f1566b0ccc None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:28:47.099739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:47.147833 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203db964-99ea-46ce-b35e-4b29b0ac036e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:28:47.148369 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203db964-99ea-46ce-b35e-4b29b0ac036e service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:47.148644 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203db964-99ea-46ce-b35e-4b29b0ac036e service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:47.148944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203db964-99ea-46ce-b35e-4b29b0ac036e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:47.150582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-203db964-99ea-46ce-b35e-4b29b0ac036e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:47.249052 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203db964-99ea-46ce-b35e-4b29b0ac036e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:28:47.250691 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203db964-99ea-46ce-b35e-4b29b0ac036e service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:28:47.271990 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 530/1062] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:28:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:28:49.477069 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80628e-d8e3-4982-b56c-086e24be8861 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:49.477682 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80628e-d8e3-4982-b56c-086e24be8861 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:49.477682 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80628e-d8e3-4982-b56c-086e24be8861 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:49.479025 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 533/1063] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:28:49.484392 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54645fd-284b-4446-8e9d-088683ebc80a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:49.484851 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54645fd-284b-4446-8e9d-088683ebc80a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:49.485071 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54645fd-284b-4446-8e9d-088683ebc80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:49.557446 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f54645fd-284b-4446-8e9d-088683ebc80a None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:49.558665 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f54645fd-284b-4446-8e9d-088683ebc80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:49.595593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:49.668998 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 531/1064] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:28:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:28:49.975364 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80628e-d8e3-4982-b56c-086e24be8861 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:28:49.977425 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-dd80628e-d8e3-4982-b56c-086e24be8861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:50.068201 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:50.141297 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80628e-d8e3-4982-b56c-086e24be8861 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:28:50.262012 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf8f50-3457-4646-8f5d-b695c21aaa31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:28:50.262487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf8f50-3457-4646-8f5d-b695c21aaa31 service nova] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:50.262770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf8f50-3457-4646-8f5d-b695c21aaa31 service nova] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:50.263044 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf8f50-3457-4646-8f5d-b695c21aaa31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:50.264655 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-44bf8f50-3457-4646-8f5d-b695c21aaa31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:50.303714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:50.357346 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf8f50-3457-4646-8f5d-b695c21aaa31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:28:50.359010 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf8f50-3457-4646-8f5d-b695c21aaa31 service nova] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:28:50.378459 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 534/1065] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:28:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:28:52.615317 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad408f3a-5659-414f-b602-cd8da802f985 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:52.615873 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad408f3a-5659-414f-b602-cd8da802f985 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:52.615873 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad408f3a-5659-414f-b602-cd8da802f985 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:52.617242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 532/1066] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:28:52.622636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ff62c-8b9b-4322-8d71-044035228296 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:52.622636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ff62c-8b9b-4322-8d71-044035228296 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:52.623285 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ff62c-8b9b-4322-8d71-044035228296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:52.694123 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-b07ff62c-8b9b-4322-8d71-044035228296 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:52.695335 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b07ff62c-8b9b-4322-8d71-044035228296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:52.797075 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 535/1067] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:28:53.085873 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad408f3a-5659-414f-b602-cd8da802f985 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:28:53.087393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ad408f3a-5659-414f-b602-cd8da802f985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:53.235999 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad408f3a-5659-414f-b602-cd8da802f985 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:28:53.252736 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:53.349553 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b5942-0499-4528-bc71-587133fc8419 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:28:53.350076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b5942-0499-4528-bc71-587133fc8419 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:53.350355 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b5942-0499-4528-bc71-587133fc8419 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:53.350626 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b5942-0499-4528-bc71-587133fc8419 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:53.352198 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a34b5942-0499-4528-bc71-587133fc8419 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:53.443385 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b5942-0499-4528-bc71-587133fc8419 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:28:53.445041 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b5942-0499-4528-bc71-587133fc8419 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:28:53.463626 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 533/1068] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:28:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:28:53.814402 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97ba65-55a0-42f4-acbe-a124984ebe66 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:53.814959 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97ba65-55a0-42f4-acbe-a124984ebe66 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:53.814959 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97ba65-55a0-42f4-acbe-a124984ebe66 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:53.816507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 536/1069] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:28:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:28:53.821960 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e939a3a-d2eb-420b-bd06-feef3f1c57fb None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:53.822383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e939a3a-d2eb-420b-bd06-feef3f1c57fb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:53.822629 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e939a3a-d2eb-420b-bd06-feef3f1c57fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:53.849556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:53.893141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9e939a3a-d2eb-420b-bd06-feef3f1c57fb None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:53.894348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9e939a3a-d2eb-420b-bd06-feef3f1c57fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:53.993873 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 534/1070] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:28:54.294363 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98c1dd-d800-4790-a286-c61b4e66f29b None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:54.294714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98c1dd-d800-4790-a286-c61b4e66f29b None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:54.295095 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98c1dd-d800-4790-a286-c61b4e66f29b None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:54.296560 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 537/1071] 10.222.0.23 () {54 vars in 962 bytes} [Thu Oct 20 13:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:28:54.302518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a224f-4536-4d7e-9589-d962e330b0b9 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:54.302801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a224f-4536-4d7e-9589-d962e330b0b9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:54.303071 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a224f-4536-4d7e-9589-d962e330b0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:54.381440 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-027a224f-4536-4d7e-9589-d962e330b0b9 None None] MFA Rules not processed for user `87b5093e963c4753928a8c6091afc52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:54.382649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-027a224f-4536-4d7e-9589-d962e330b0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:54.424626 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:54.460790 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 535/1072] 10.222.0.23 () {58 vars in 1059 bytes} [Thu Oct 20 13:28:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:28:54.470271 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98c1dd-d800-4790-a286-c61b4e66f29b None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:28:54.471830 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-4e98c1dd-d800-4790-a286-c61b4e66f29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:54.485727 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:54.585608 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98c1dd-d800-4790-a286-c61b4e66f29b None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:28:54.664076 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd97af4-eebd-4b1f-ba27-ee441efd0168 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87b5093e963c4753928a8c6091afc52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:28:54.664676 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd97af4-eebd-4b1f-ba27-ee441efd0168 service glance] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:54.664977 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd97af4-eebd-4b1f-ba27-ee441efd0168 service glance] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:54.665243 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd97af4-eebd-4b1f-ba27-ee441efd0168 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:54.666845 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-bfd97af4-eebd-4b1f-ba27-ee441efd0168 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:54.737063 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:54.780308 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd97af4-eebd-4b1f-ba27-ee441efd0168 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:28:54.781797 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd97af4-eebd-4b1f-ba27-ee441efd0168 service glance] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:28:54.801363 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 538/1073] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:28:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:28:55.542438 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733c7f3-91de-4a44-b9e9-c91710f7a193 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:55.543058 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733c7f3-91de-4a44-b9e9-c91710f7a193 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:55.543058 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733c7f3-91de-4a44-b9e9-c91710f7a193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:55.617448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-3733c7f3-91de-4a44-b9e9-c91710f7a193 None None] MFA Rules not processed for user `85bb137d0fb74031a49a5c629b32c32b`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:55.618628 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3733c7f3-91de-4a44-b9e9-c91710f7a193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:55.659250 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd87ecf-dab4-4485-8202-8d6c8ab01a0e None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:55.659859 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd87ecf-dab4-4485-8202-8d6c8ab01a0e None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:55.659859 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd87ecf-dab4-4485-8202-8d6c8ab01a0e None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:55.661156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 539/1074] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:28:55.666080 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389d65c-2fbc-4052-86dc-0a8eaa27d3e8 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:55.666362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389d65c-2fbc-4052-86dc-0a8eaa27d3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:55.666640 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389d65c-2fbc-4052-86dc-0a8eaa27d3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:55.707201 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 536/1075] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 20 13:28:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:28:55.734360 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-2389d65c-2fbc-4052-86dc-0a8eaa27d3e8 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:55.736093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-2389d65c-2fbc-4052-86dc-0a8eaa27d3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:55.751258 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ca526-33a8-462c-b422-4456dacf70cb None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:55.751649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ca526-33a8-462c-b422-4456dacf70cb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:55.752126 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ca526-33a8-462c-b422-4456dacf70cb None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:55.753628 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 537/1076] 10.222.0.23 () {54 vars in 943 bytes} [Thu Oct 20 13:28:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:28:55.759603 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612436b0-45c7-4229-ae13-a07a89787b29 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:55.759889 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612436b0-45c7-4229-ae13-a07a89787b29 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:55.760163 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612436b0-45c7-4229-ae13-a07a89787b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:55.831073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:55.837039 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 540/1077] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:28:55.838968 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-612436b0-45c7-4229-ae13-a07a89787b29 None None] MFA Rules not processed for user `9834211488354651803000c7e8fab80b`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:55.840163 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-612436b0-45c7-4229-ae13-a07a89787b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:55.948236 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 538/1078] 10.222.0.23 () {58 vars in 1040 bytes} [Thu Oct 20 13:28:55 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:28:55.958211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389d65c-2fbc-4052-86dc-0a8eaa27d3e8 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:28:55.959437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-2389d65c-2fbc-4052-86dc-0a8eaa27d3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:55.987988 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:56.064776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389d65c-2fbc-4052-86dc-0a8eaa27d3e8 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:28:56.133123 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612436b0-45c7-4229-ae13-a07a89787b29 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:28:56.134352 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-612436b0-45c7-4229-ae13-a07a89787b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:56.174662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db363fb3-f266-4a02-b1b8-bca7bf6f139d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9834211488354651803000c7e8fab80b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:28:56.175434 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db363fb3-f266-4a02-b1b8-bca7bf6f139d service swift] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:56.175718 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db363fb3-f266-4a02-b1b8-bca7bf6f139d service swift] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:56.175987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db363fb3-f266-4a02-b1b8-bca7bf6f139d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:56.178116 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-db363fb3-f266-4a02-b1b8-bca7bf6f139d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:56.219053 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:56.231871 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612436b0-45c7-4229-ae13-a07a89787b29 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:28:56.254078 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:56.265298 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db363fb3-f266-4a02-b1b8-bca7bf6f139d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:28:56.266953 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db363fb3-f266-4a02-b1b8-bca7bf6f139d service swift] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:28:56.275324 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 541/1079] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 20 13:28:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 13:28:56.352845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801d542-980a-4230-8c43-5102a5d21f11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:28:56.353641 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801d542-980a-4230-8c43-5102a5d21f11 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:56.353743 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801d542-980a-4230-8c43-5102a5d21f11 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:56.354016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801d542-980a-4230-8c43-5102a5d21f11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:56.355619 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e801d542-980a-4230-8c43-5102a5d21f11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:56.469447 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e801d542-980a-4230-8c43-5102a5d21f11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:28:56.471488 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e801d542-980a-4230-8c43-5102a5d21f11 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:28:56.489837 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 539/1080] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:28:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:28:58.716740 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43daaf-1f23-482e-80fd-9710cb6ae3e5 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:58.717352 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43daaf-1f23-482e-80fd-9710cb6ae3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:58.717352 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43daaf-1f23-482e-80fd-9710cb6ae3e5 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:58.718768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 542/1081] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:28:58.725045 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305299c2-dcb9-4e66-98ca-3ab1c81f034d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:58.725611 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305299c2-dcb9-4e66-98ca-3ab1c81f034d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:58.725611 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305299c2-dcb9-4e66-98ca-3ab1c81f034d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:58.760362 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:58.803565 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-305299c2-dcb9-4e66-98ca-3ab1c81f034d None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:28:58.804784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-305299c2-dcb9-4e66-98ca-3ab1c81f034d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:58.910967 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 540/1082] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:28:59.199697 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43daaf-1f23-482e-80fd-9710cb6ae3e5 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:28:59.201246 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-0a43daaf-1f23-482e-80fd-9710cb6ae3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:59.370884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43daaf-1f23-482e-80fd-9710cb6ae3e5 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:28:59.443449 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:28:59.548287 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ff237-2dee-4bd3-993d-c1b164981151 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:28:59.548919 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ff237-2dee-4bd3-993d-c1b164981151 service nova] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:28:59.549202 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ff237-2dee-4bd3-993d-c1b164981151 service nova] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:28:59.549475 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ff237-2dee-4bd3-993d-c1b164981151 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:28:59.551102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b75ff237-2dee-4bd3-993d-c1b164981151 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:28:59.665325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ff237-2dee-4bd3-993d-c1b164981151 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:28:59.667125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ff237-2dee-4bd3-993d-c1b164981151 service nova] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:28:59.693181 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 543/1083] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:28:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:29:01.941376 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acdfd0-ea0b-45f2-a2bc-6605a9a64c6f None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:01.941973 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acdfd0-ea0b-45f2-a2bc-6605a9a64c6f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:01.941973 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acdfd0-ea0b-45f2-a2bc-6605a9a64c6f None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:01.943256 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 541/1084] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:29:01.948543 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb757556-f59a-4ee9-ac80-e6df03c75843 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:01.949046 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb757556-f59a-4ee9-ac80-e6df03c75843 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:01.949148 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb757556-f59a-4ee9-ac80-e6df03c75843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:02.005143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:02.026859 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-bb757556-f59a-4ee9-ac80-e6df03c75843 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:29:02.026859 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-bb757556-f59a-4ee9-ac80-e6df03c75843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:02.141847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 544/1085] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:29:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:29:02.424425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acdfd0-ea0b-45f2-a2bc-6605a9a64c6f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:29:02.426025 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-81acdfd0-ea0b-45f2-a2bc-6605a9a64c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:02.536036 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:02.589331 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acdfd0-ea0b-45f2-a2bc-6605a9a64c6f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:29:02.732973 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587cd667-cf92-4de1-b971-b9b74646b315 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:29:02.733493 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cd667-cf92-4de1-b971-b9b74646b315 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:02.733777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cd667-cf92-4de1-b971-b9b74646b315 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:02.734111 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587cd667-cf92-4de1-b971-b9b74646b315 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:02.736322 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-587cd667-cf92-4de1-b971-b9b74646b315 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:02.813112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:02.842240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587cd667-cf92-4de1-b971-b9b74646b315 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:29:02.844431 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587cd667-cf92-4de1-b971-b9b74646b315 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:29:02.866230 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 542/1086] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:29:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:29:12.136387 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=47 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:12.863292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=16 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:18.777916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1186bf4-20cf-4ce8-b846-f7f368bf8042 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:18.778429 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1186bf4-20cf-4ce8-b846-f7f368bf8042 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:18.778429 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1186bf4-20cf-4ce8-b846-f7f368bf8042 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:18.779775 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 545/1087] 10.222.0.23 () {54 vars in 918 bytes} [Thu Oct 20 13:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:29:18.785011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1295aa-3419-4985-983b-fcc011a666c1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:18.785412 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1295aa-3419-4985-983b-fcc011a666c1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:18.785555 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1295aa-3419-4985-983b-fcc011a666c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:18.864174 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-de1295aa-3419-4985-983b-fcc011a666c1 None None] MFA Rules not processed for user `2440594934b64621a01c441133778155`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:29:18.865396 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-de1295aa-3419-4985-983b-fcc011a666c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:18.970358 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 543/1088] 10.222.0.23 () {58 vars in 1015 bytes} [Thu Oct 20 13:29:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:29:18.984822 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799742e-e64c-40c9-9264-92fc5c956fb8 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:18.985143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799742e-e64c-40c9-9264-92fc5c956fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:18.985417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799742e-e64c-40c9-9264-92fc5c956fb8 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:18.986746 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 546/1089] 10.222.0.23 () {54 vars in 918 bytes} [Thu Oct 20 13:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:29:18.991182 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa8d8d-89c7-4525-bac2-c36179786e24 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:18.991482 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa8d8d-89c7-4525-bac2-c36179786e24 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:18.991738 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa8d8d-89c7-4525-bac2-c36179786e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:19.063972 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:19.066871 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-ebaa8d8d-89c7-4525-bac2-c36179786e24 None None] MFA Rules not processed for user `ee5c2a3855cb4a668177aa19f337778d`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:29:19.068520 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ebaa8d8d-89c7-4525-bac2-c36179786e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:19.182865 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 544/1090] 10.222.0.23 () {58 vars in 1015 bytes} [Thu Oct 20 13:29:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:29:21.768056 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ddda79-0cd5-4e7f-a4b6-ca8de5c06f15 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:21.768564 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ddda79-0cd5-4e7f-a4b6-ca8de5c06f15 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:21.768564 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ddda79-0cd5-4e7f-a4b6-ca8de5c06f15 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:21.769955 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 547/1091] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 13:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:29:21.775169 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63793743-6241-48a9-ade9-10d23b39670f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:21.775606 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63793743-6241-48a9-ade9-10d23b39670f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:21.775725 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63793743-6241-48a9-ade9-10d23b39670f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:21.841276 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-63793743-6241-48a9-ade9-10d23b39670f None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:29:21.842468 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-63793743-6241-48a9-ade9-10d23b39670f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:21.897262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:21.938353 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 545/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 13:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:29:22.228736 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ddda79-0cd5-4e7f-a4b6-ca8de5c06f15 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:29:22.230361 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-39ddda79-0cd5-4e7f-a4b6-ca8de5c06f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:22.372109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ddda79-0cd5-4e7f-a4b6-ca8de5c06f15 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:29:22.442815 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:22.481515 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9687370-5c3a-4f30-a998-9c76fbfaeccf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:29:22.482034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9687370-5c3a-4f30-a998-9c76fbfaeccf service nova] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:22.482309 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9687370-5c3a-4f30-a998-9c76fbfaeccf service nova] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:22.482579 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9687370-5c3a-4f30-a998-9c76fbfaeccf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:22.484832 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-d9687370-5c3a-4f30-a998-9c76fbfaeccf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:22.575048 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9687370-5c3a-4f30-a998-9c76fbfaeccf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:29:22.576642 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9687370-5c3a-4f30-a998-9c76fbfaeccf service nova] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:29:22.594997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 548/1093] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:29:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:29:23.740872 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe302179-be03-4dab-ae72-33012e824c99 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:23.741407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe302179-be03-4dab-ae72-33012e824c99 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:23.741407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe302179-be03-4dab-ae72-33012e824c99 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:23.742691 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 546/1094] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 13:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:29:23.749553 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1803a-14b0-42e5-82c9-1160dadd1ac5 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:23.750065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1803a-14b0-42e5-82c9-1160dadd1ac5 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:23.750163 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1803a-14b0-42e5-82c9-1160dadd1ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:23.824982 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-ffb1803a-14b0-42e5-82c9-1160dadd1ac5 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:29:23.826189 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-ffb1803a-14b0-42e5-82c9-1160dadd1ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:23.852435 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:23.921410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 549/1095] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 13:29:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:29:24.069885 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe302179-be03-4dab-ae72-33012e824c99 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:29:24.071406 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fe302179-be03-4dab-ae72-33012e824c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:24.215002 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe302179-be03-4dab-ae72-33012e824c99 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:29:24.254341 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:24.325576 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756a8e7-a205-476d-aa5c-0b42af662634 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:29:24.326095 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756a8e7-a205-476d-aa5c-0b42af662634 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:24.326371 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756a8e7-a205-476d-aa5c-0b42af662634 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:24.326634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756a8e7-a205-476d-aa5c-0b42af662634 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:24.328891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0756a8e7-a205-476d-aa5c-0b42af662634 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:24.416217 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0756a8e7-a205-476d-aa5c-0b42af662634 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:29:24.417825 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0756a8e7-a205-476d-aa5c-0b42af662634 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:29:24.435788 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 547/1096] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:29:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:29:25.577257 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8df57-a11e-4abd-bea6-e3f06e503a41 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:25.577783 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8df57-a11e-4abd-bea6-e3f06e503a41 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:25.577879 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8df57-a11e-4abd-bea6-e3f06e503a41 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:25.579191 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 550/1097] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 13:29:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:29:25.585489 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad22153-7efd-469a-97ea-35b81a227747 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:25.585914 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad22153-7efd-469a-97ea-35b81a227747 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:25.586047 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad22153-7efd-469a-97ea-35b81a227747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:25.640421 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:25.662440 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-cad22153-7efd-469a-97ea-35b81a227747 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:29:25.663639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cad22153-7efd-469a-97ea-35b81a227747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:25.763979 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 548/1098] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 13:29:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:29:25.916322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8df57-a11e-4abd-bea6-e3f06e503a41 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:29:25.917884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-7eb8df57-a11e-4abd-bea6-e3f06e503a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:26.061886 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8df57-a11e-4abd-bea6-e3f06e503a41 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:29:26.073904 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:26.176371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7503b39-da96-47e3-972b-27ba7810174e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:29:26.177007 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7503b39-da96-47e3-972b-27ba7810174e service nova] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:26.177286 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7503b39-da96-47e3-972b-27ba7810174e service nova] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:26.177553 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7503b39-da96-47e3-972b-27ba7810174e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:26.179137 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-d7503b39-da96-47e3-972b-27ba7810174e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:26.271271 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7503b39-da96-47e3-972b-27ba7810174e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:29:26.272905 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7503b39-da96-47e3-972b-27ba7810174e service nova] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:29:26.291192 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 551/1099] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:29:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:29:27.403830 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a3076-7894-45a0-9929-02a545d4bb36 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:27.404373 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a3076-7894-45a0-9929-02a545d4bb36 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:27.404373 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a3076-7894-45a0-9929-02a545d4bb36 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:27.405815 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 549/1100] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 13:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:29:27.412647 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3c278-f48a-4600-9b34-fe79c8036103 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:27.413130 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3c278-f48a-4600-9b34-fe79c8036103 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:27.413232 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3c278-f48a-4600-9b34-fe79c8036103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:27.432367 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:27.488076 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-78e3c278-f48a-4600-9b34-fe79c8036103 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:29:27.489299 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-78e3c278-f48a-4600-9b34-fe79c8036103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:27.584638 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 552/1101] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 13:29:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:29:27.736605 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a3076-7894-45a0-9929-02a545d4bb36 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:29:27.738074 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7a8a3076-7894-45a0-9929-02a545d4bb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:27.861149 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:27.897235 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a3076-7894-45a0-9929-02a545d4bb36 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:29:28.007056 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42468e-985f-4263-b410-86ae97f23037 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:29:28.007576 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42468e-985f-4263-b410-86ae97f23037 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:28.007853 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42468e-985f-4263-b410-86ae97f23037 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:28.008120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42468e-985f-4263-b410-86ae97f23037 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:28.009719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4f42468e-985f-4263-b410-86ae97f23037 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:28.085709 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:28.101085 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42468e-985f-4263-b410-86ae97f23037 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:29:28.103030 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42468e-985f-4263-b410-86ae97f23037 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:29:28.122805 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 550/1102] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:29:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:29:29.232334 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f157736-7cee-432e-9105-3fef09818d10 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:29.233026 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f157736-7cee-432e-9105-3fef09818d10 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:29.233026 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f157736-7cee-432e-9105-3fef09818d10 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:29.234209 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 553/1103] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 13:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:29:29.240529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155e5b5-6663-411f-acf4-355021e427c1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:29.241018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155e5b5-6663-411f-acf4-355021e427c1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:29.241112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155e5b5-6663-411f-acf4-355021e427c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:29.317333 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-4155e5b5-6663-411f-acf4-355021e427c1 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:29:29.318540 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4155e5b5-6663-411f-acf4-355021e427c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:29.412791 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 551/1104] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 13:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:29:29.564348 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f157736-7cee-432e-9105-3fef09818d10 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:29:29.565923 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-9f157736-7cee-432e-9105-3fef09818d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:29.649919 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:29.714081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f157736-7cee-432e-9105-3fef09818d10 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:29:29.829001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788f92a-4f4d-4bec-ace2-c86570686660 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:29:29.829601 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788f92a-4f4d-4bec-ace2-c86570686660 service nova] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:29.829876 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788f92a-4f4d-4bec-ace2-c86570686660 service nova] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:29.830144 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788f92a-4f4d-4bec-ace2-c86570686660 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:29.831733 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-4788f92a-4f4d-4bec-ace2-c86570686660 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:29.880808 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:29.922292 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788f92a-4f4d-4bec-ace2-c86570686660 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:29:29.923884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788f92a-4f4d-4bec-ace2-c86570686660 service nova] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:29:29.942268 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 554/1105] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:29:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:29:30.983784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2716de-2da2-4ec6-871a-04df20a2adef None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:30.984326 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2716de-2da2-4ec6-871a-04df20a2adef None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:30.984326 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2716de-2da2-4ec6-871a-04df20a2adef None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:30.985706 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 552/1106] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 13:29:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:29:30.991745 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5e815-45d3-4ea2-bf73-574306351b6e None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:30.992186 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5e815-45d3-4ea2-bf73-574306351b6e None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:30.992282 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5e815-45d3-4ea2-bf73-574306351b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:31.066194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-7fc5e815-45d3-4ea2-bf73-574306351b6e None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:29:31.067378 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-7fc5e815-45d3-4ea2-bf73-574306351b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:31.161202 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 555/1107] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 13:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:29:31.311112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2716de-2da2-4ec6-871a-04df20a2adef None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:29:31.312637 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ff2716de-2da2-4ec6-871a-04df20a2adef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:31.360811 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:31.463698 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2716de-2da2-4ec6-871a-04df20a2adef None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:29:31.579489 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792dd69-00cd-4ead-8d82-8aef59c85355 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:29:31.580011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792dd69-00cd-4ead-8d82-8aef59c85355 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:31.580284 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792dd69-00cd-4ead-8d82-8aef59c85355 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:31.580552 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792dd69-00cd-4ead-8d82-8aef59c85355 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:31.582160 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0792dd69-00cd-4ead-8d82-8aef59c85355 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:31.588896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:31.669678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792dd69-00cd-4ead-8d82-8aef59c85355 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:29:31.671983 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792dd69-00cd-4ead-8d82-8aef59c85355 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:29:31.689503 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 553/1108] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:29:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:29:32.745274 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3dda2-6139-4ceb-b9fb-8316655e5362 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:32.745841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3dda2-6139-4ceb-b9fb-8316655e5362 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:32.745841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3dda2-6139-4ceb-b9fb-8316655e5362 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:32.747151 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 556/1109] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 13:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:29:32.754387 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62c745-497f-4ba8-bea6-f0ef2d1bd3cd None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:32.754855 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62c745-497f-4ba8-bea6-f0ef2d1bd3cd None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:32.754961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62c745-497f-4ba8-bea6-f0ef2d1bd3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:32.830653 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2c62c745-497f-4ba8-bea6-f0ef2d1bd3cd None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:29:32.831843 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2c62c745-497f-4ba8-bea6-f0ef2d1bd3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:32.899306 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:32.929091 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 554/1110] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 13:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:29:33.081062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb3dda2-6139-4ceb-b9fb-8316655e5362 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:29:33.082551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-7fb3dda2-6139-4ceb-b9fb-8316655e5362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:33.099974 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:33.235888 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb3dda2-6139-4ceb-b9fb-8316655e5362 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:29:33.320746 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:33.348114 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340650c-8d4b-4b02-ba9b-315cc6f9a3ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:29:33.348650 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340650c-8d4b-4b02-ba9b-315cc6f9a3ae service nova] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:33.348956 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340650c-8d4b-4b02-ba9b-315cc6f9a3ae service nova] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:33.349231 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340650c-8d4b-4b02-ba9b-315cc6f9a3ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:33.350822 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-c340650c-8d4b-4b02-ba9b-315cc6f9a3ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:33.438286 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340650c-8d4b-4b02-ba9b-315cc6f9a3ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:29:33.439911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340650c-8d4b-4b02-ba9b-315cc6f9a3ae service nova] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:29:33.457960 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 557/1111] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:29:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:29:34.506794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd8214-00af-4ad1-a14f-c27693356211 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:34.507367 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd8214-00af-4ad1-a14f-c27693356211 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:34.507367 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd8214-00af-4ad1-a14f-c27693356211 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:34.508696 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 555/1112] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 13:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:29:34.515833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469e803-892a-4d8e-9f24-7c5b1aeedb48 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:34.516285 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469e803-892a-4d8e-9f24-7c5b1aeedb48 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:34.516395 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469e803-892a-4d8e-9f24-7c5b1aeedb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:34.590275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-6469e803-892a-4d8e-9f24-7c5b1aeedb48 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:29:34.591462 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-6469e803-892a-4d8e-9f24-7c5b1aeedb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:34.632755 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:34.686795 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 558/1113] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 13:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:29:34.836755 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd8214-00af-4ad1-a14f-c27693356211 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:29:34.838292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cbcd8214-00af-4ad1-a14f-c27693356211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:34.984067 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd8214-00af-4ad1-a14f-c27693356211 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:29:35.039424 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:35.098660 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fabb45-4b50-4c4d-8832-74ac5203d2be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:29:35.099189 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fabb45-4b50-4c4d-8832-74ac5203d2be service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:35.099494 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fabb45-4b50-4c4d-8832-74ac5203d2be service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:35.099766 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fabb45-4b50-4c4d-8832-74ac5203d2be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:35.101401 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-12fabb45-4b50-4c4d-8832-74ac5203d2be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:35.189987 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fabb45-4b50-4c4d-8832-74ac5203d2be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:29:35.191568 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fabb45-4b50-4c4d-8832-74ac5203d2be service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:29:35.209470 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 556/1114] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:29:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:29:36.272756 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca7782-9692-4be4-a157-3754fc14717c None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:36.273428 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca7782-9692-4be4-a157-3754fc14717c None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:36.273428 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca7782-9692-4be4-a157-3754fc14717c None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:36.274704 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 559/1115] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 13:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:29:36.281040 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e24072-b329-465c-8514-9691b564f51f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:36.281500 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e24072-b329-465c-8514-9691b564f51f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:36.281611 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e24072-b329-465c-8514-9691b564f51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:36.356096 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-56e24072-b329-465c-8514-9691b564f51f None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:29:36.357920 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-56e24072-b329-465c-8514-9691b564f51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:36.359972 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:36.454701 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 557/1116] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 13:29:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:29:36.608155 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca7782-9692-4be4-a157-3754fc14717c None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:29:36.609738 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-79ca7782-9692-4be4-a157-3754fc14717c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:36.754910 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca7782-9692-4be4-a157-3754fc14717c None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:29:36.782959 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:36.866068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f4f82-15da-4fd8-b134-707b0108a8d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:29:36.866583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f4f82-15da-4fd8-b134-707b0108a8d0 service nova] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:36.866855 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f4f82-15da-4fd8-b134-707b0108a8d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:36.867122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f4f82-15da-4fd8-b134-707b0108a8d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:36.868699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-ac8f4f82-15da-4fd8-b134-707b0108a8d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:36.958374 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8f4f82-15da-4fd8-b134-707b0108a8d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:29:36.959981 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8f4f82-15da-4fd8-b134-707b0108a8d0 service nova] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:29:36.978809 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 560/1117] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:29:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:29:39.066369 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f679ad-11cd-4bae-bc3f-818289049346 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:39.066938 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f679ad-11cd-4bae-bc3f-818289049346 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:39.066938 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f679ad-11cd-4bae-bc3f-818289049346 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:39.068254 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 558/1118] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 13:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:29:39.073485 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b12e06-d509-4b94-9dad-fe97a333ce80 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:39.073883 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b12e06-d509-4b94-9dad-fe97a333ce80 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:39.074045 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b12e06-d509-4b94-9dad-fe97a333ce80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:39.118278 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:39.147222 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-45b12e06-d509-4b94-9dad-fe97a333ce80 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:29:39.148426 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-45b12e06-d509-4b94-9dad-fe97a333ce80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:39.252375 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 561/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 13:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:29:39.539395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f679ad-11cd-4bae-bc3f-818289049346 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:29:39.540995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a9f679ad-11cd-4bae-bc3f-818289049346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:39.644899 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f679ad-11cd-4bae-bc3f-818289049346 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:29:39.676987 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:39.720302 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c1c71-bcb3-4b79-9d32-c111dd82fbb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:29:39.720815 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c1c71-bcb3-4b79-9d32-c111dd82fbb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:39.721157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c1c71-bcb3-4b79-9d32-c111dd82fbb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:39.721410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c1c71-bcb3-4b79-9d32-c111dd82fbb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:39.722983 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-644c1c71-bcb3-4b79-9d32-c111dd82fbb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:39.829378 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c1c71-bcb3-4b79-9d32-c111dd82fbb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:29:39.830812 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c1c71-bcb3-4b79-9d32-c111dd82fbb7 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:29:39.849039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 559/1120] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:29:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:29:42.129742 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1fec1-cc2f-42ad-ae17-06d8668bd358 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:42.130370 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1fec1-cc2f-42ad-ae17-06d8668bd358 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:42.130370 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1fec1-cc2f-42ad-ae17-06d8668bd358 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:42.131804 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 562/1121] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 13:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:29:42.137273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861819dc-4cc4-4a3c-849e-a7e16f15eb1e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:42.137695 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861819dc-4cc4-4a3c-849e-a7e16f15eb1e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:42.137833 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861819dc-4cc4-4a3c-849e-a7e16f15eb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:42.204038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-861819dc-4cc4-4a3c-849e-a7e16f15eb1e None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:29:42.205302 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-861819dc-4cc4-4a3c-849e-a7e16f15eb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:42.218011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:42.307610 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 560/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 13:29:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:29:42.595929 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1fec1-cc2f-42ad-ae17-06d8668bd358 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:29:42.597501 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-81c1fec1-cc2f-42ad-ae17-06d8668bd358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:42.701119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1fec1-cc2f-42ad-ae17-06d8668bd358 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:29:42.713762 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:42.776090 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b027c-f445-47f5-b6b4-ad2cc65c6cbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:29:42.776704 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b027c-f445-47f5-b6b4-ad2cc65c6cbb service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:42.777016 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b027c-f445-47f5-b6b4-ad2cc65c6cbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:42.777287 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b027c-f445-47f5-b6b4-ad2cc65c6cbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:42.779421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-f39b027c-f445-47f5-b6b4-ad2cc65c6cbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:42.887946 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b027c-f445-47f5-b6b4-ad2cc65c6cbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:29:42.889454 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b027c-f445-47f5-b6b4-ad2cc65c6cbb service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:29:42.907580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 563/1123] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:29:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:29:45.450590 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdb627-bf06-4beb-8c3b-9e97c1c4acd6 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:45.451685 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdb627-bf06-4beb-8c3b-9e97c1c4acd6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:45.451685 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdb627-bf06-4beb-8c3b-9e97c1c4acd6 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:45.453191 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 561/1124] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 13:29:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:29:45.458548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2173c33-5cdd-4785-8b04-6291518e6cc0 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:45.459001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2173c33-5cdd-4785-8b04-6291518e6cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:45.459095 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2173c33-5cdd-4785-8b04-6291518e6cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:45.508126 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:45.526433 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-c2173c33-5cdd-4785-8b04-6291518e6cc0 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:29:45.527618 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-c2173c33-5cdd-4785-8b04-6291518e6cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:45.621668 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 564/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 13:29:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:29:45.909987 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebdb627-bf06-4beb-8c3b-9e97c1c4acd6 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:29:45.911524 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-aebdb627-bf06-4beb-8c3b-9e97c1c4acd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:46.014319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebdb627-bf06-4beb-8c3b-9e97c1c4acd6 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:29:46.054096 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:46.087756 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7cec2-5d3c-41d9-b12c-3bdd2e95b7ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:29:46.088350 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7cec2-5d3c-41d9-b12c-3bdd2e95b7ee service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:46.088625 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7cec2-5d3c-41d9-b12c-3bdd2e95b7ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:46.088917 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7cec2-5d3c-41d9-b12c-3bdd2e95b7ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:46.090483 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-eec7cec2-5d3c-41d9-b12c-3bdd2e95b7ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:46.198002 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7cec2-5d3c-41d9-b12c-3bdd2e95b7ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:29:46.199351 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7cec2-5d3c-41d9-b12c-3bdd2e95b7ee service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:29:46.218282 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 562/1126] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:29:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:29:48.465308 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83abeb-8323-444d-9830-c98cf9fb77d1 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:48.465797 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83abeb-8323-444d-9830-c98cf9fb77d1 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:48.465934 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83abeb-8323-444d-9830-c98cf9fb77d1 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:48.467830 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 565/1127] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 13:29:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:29:48.473360 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f9436-14c3-4c15-8d8e-bc6719d3cf86 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:48.473744 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f9436-14c3-4c15-8d8e-bc6719d3cf86 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:48.473909 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f9436-14c3-4c15-8d8e-bc6719d3cf86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:48.541920 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-607f9436-14c3-4c15-8d8e-bc6719d3cf86 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:29:48.543128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-607f9436-14c3-4c15-8d8e-bc6719d3cf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:48.563421 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:48.637464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 563/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 13:29:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:29:48.922313 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83abeb-8323-444d-9830-c98cf9fb77d1 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:29:48.923840 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-1f83abeb-8323-444d-9830-c98cf9fb77d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:49.026796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83abeb-8323-444d-9830-c98cf9fb77d1 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:29:49.048277 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:49.101153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a46dc-3451-476c-83fb-f7b633fa1ed8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:29:49.101753 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a46dc-3451-476c-83fb-f7b633fa1ed8 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:49.102033 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a46dc-3451-476c-83fb-f7b633fa1ed8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:49.102307 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a46dc-3451-476c-83fb-f7b633fa1ed8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:49.103890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-921a46dc-3451-476c-83fb-f7b633fa1ed8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:49.212857 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a46dc-3451-476c-83fb-f7b633fa1ed8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:29:49.214232 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a46dc-3451-476c-83fb-f7b633fa1ed8 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:29:49.232651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 566/1129] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:29:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:29:51.542602 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65e4ba-3118-4c91-82d9-f1fe8f30e788 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:51.543108 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65e4ba-3118-4c91-82d9-f1fe8f30e788 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:51.543204 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65e4ba-3118-4c91-82d9-f1fe8f30e788 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:51.544481 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 564/1130] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 13:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:29:51.549899 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee19ee4-bfc6-42b8-85f5-ea34eb511b05 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:51.550346 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee19ee4-bfc6-42b8-85f5-ea34eb511b05 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:51.550453 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee19ee4-bfc6-42b8-85f5-ea34eb511b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:51.612904 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:51.615690 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-3ee19ee4-bfc6-42b8-85f5-ea34eb511b05 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:29:51.617426 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-3ee19ee4-bfc6-42b8-85f5-ea34eb511b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:51.711627 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 567/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 13:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:29:51.995308 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65e4ba-3118-4c91-82d9-f1fe8f30e788 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:29:51.996857 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3c65e4ba-3118-4c91-82d9-f1fe8f30e788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:52.099487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65e4ba-3118-4c91-82d9-f1fe8f30e788 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:29:52.149436 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:52.174424 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db2ed1-8ca1-4748-9611-e09bd6d8abe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:29:52.175027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db2ed1-8ca1-4748-9611-e09bd6d8abe3 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:52.175309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db2ed1-8ca1-4748-9611-e09bd6d8abe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:52.175586 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db2ed1-8ca1-4748-9611-e09bd6d8abe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:52.177214 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-28db2ed1-8ca1-4748-9611-e09bd6d8abe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:52.285797 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db2ed1-8ca1-4748-9611-e09bd6d8abe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:29:52.287155 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db2ed1-8ca1-4748-9611-e09bd6d8abe3 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:29:52.304714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 565/1132] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:29:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:29:54.513066 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec66b8d9-acb8-404c-a861-639bc6c8f0e7 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:54.513955 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec66b8d9-acb8-404c-a861-639bc6c8f0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:54.514534 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec66b8d9-acb8-404c-a861-639bc6c8f0e7 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:54.515906 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 568/1133] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 13:29:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:29:54.521353 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef793a2-6492-4ce4-bf1e-15c428b34b58 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:54.521765 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef793a2-6492-4ce4-bf1e-15c428b34b58 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:54.521930 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef793a2-6492-4ce4-bf1e-15c428b34b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:54.588736 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0ef793a2-6492-4ce4-bf1e-15c428b34b58 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:29:54.590006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0ef793a2-6492-4ce4-bf1e-15c428b34b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:54.620240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:54.687974 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 566/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 13:29:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:29:54.973503 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec66b8d9-acb8-404c-a861-639bc6c8f0e7 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:29:54.975039 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-ec66b8d9-acb8-404c-a861-639bc6c8f0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:55.078825 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec66b8d9-acb8-404c-a861-639bc6c8f0e7 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:29:55.107151 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:55.154053 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589fcaa-5b93-45e6-bbd9-df7e47acb661 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:29:55.155237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589fcaa-5b93-45e6-bbd9-df7e47acb661 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:55.155467 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589fcaa-5b93-45e6-bbd9-df7e47acb661 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:55.155732 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589fcaa-5b93-45e6-bbd9-df7e47acb661 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:55.157383 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-5589fcaa-5b93-45e6-bbd9-df7e47acb661 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:55.265606 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589fcaa-5b93-45e6-bbd9-df7e47acb661 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:29:55.266971 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589fcaa-5b93-45e6-bbd9-df7e47acb661 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:29:55.285857 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 569/1135] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:29:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:29:57.472754 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53ec14-12c7-4cae-bde3-70fad2913e65 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:57.473239 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53ec14-12c7-4cae-bde3-70fad2913e65 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:57.473357 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53ec14-12c7-4cae-bde3-70fad2913e65 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:57.474663 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 567/1136] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 13:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:29:57.480131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc87e05-e70b-4e26-a95e-40231fb0ed2a None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:57.480533 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc87e05-e70b-4e26-a95e-40231fb0ed2a None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:57.480688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc87e05-e70b-4e26-a95e-40231fb0ed2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:57.547538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-8cc87e05-e70b-4e26-a95e-40231fb0ed2a None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:29:57.548756 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-8cc87e05-e70b-4e26-a95e-40231fb0ed2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:57.557066 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:57.644701 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 570/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 13:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:29:57.943119 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53ec14-12c7-4cae-bde3-70fad2913e65 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:29:57.944661 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cf53ec14-12c7-4cae-bde3-70fad2913e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:58.047520 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53ec14-12c7-4cae-bde3-70fad2913e65 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:29:58.104248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:29:58.125332 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb09b59-529b-4bb3-b933-5d0adfe5a6c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:29:58.126198 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb09b59-529b-4bb3-b933-5d0adfe5a6c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:29:58.126480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb09b59-529b-4bb3-b933-5d0adfe5a6c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:29:58.126754 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb09b59-529b-4bb3-b933-5d0adfe5a6c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:29:58.128357 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0bb09b59-529b-4bb3-b933-5d0adfe5a6c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:29:58.237487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb09b59-529b-4bb3-b933-5d0adfe5a6c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:29:58.238830 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb09b59-529b-4bb3-b933-5d0adfe5a6c4 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:29:58.257387 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 568/1138] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:29:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:30:00.458199 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353eaf1-17e8-465d-bbdc-3d98af047a96 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:00.458810 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353eaf1-17e8-465d-bbdc-3d98af047a96 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:00.458810 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353eaf1-17e8-465d-bbdc-3d98af047a96 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:00.460158 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 571/1139] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 13:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:30:00.465488 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05fccd-5c77-41fb-83d4-d5acf829d2bc None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:00.465864 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05fccd-5c77-41fb-83d4-d5acf829d2bc None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:00.466030 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05fccd-5c77-41fb-83d4-d5acf829d2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:00.532452 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7c05fccd-5c77-41fb-83d4-d5acf829d2bc None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:30:00.533671 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7c05fccd-5c77-41fb-83d4-d5acf829d2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:00.571042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:30:00.629389 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 569/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 13:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:30:00.913816 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353eaf1-17e8-465d-bbdc-3d98af047a96 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:30:00.915371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-a353eaf1-17e8-465d-bbdc-3d98af047a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:01.017218 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353eaf1-17e8-465d-bbdc-3d98af047a96 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:30:01.059323 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:30:01.095488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380a833-1867-4a6c-b9d7-9845a354efe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:30:01.096087 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380a833-1867-4a6c-b9d7-9845a354efe8 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:01.096364 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380a833-1867-4a6c-b9d7-9845a354efe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:01.096635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380a833-1867-4a6c-b9d7-9845a354efe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:01.098249 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-0380a833-1867-4a6c-b9d7-9845a354efe8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:01.205693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380a833-1867-4a6c-b9d7-9845a354efe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:30:01.207054 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380a833-1867-4a6c-b9d7-9845a354efe8 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:30:01.224699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 572/1141] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:30:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:30:03.396487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e74d7b-ef31-49e8-944f-219310238af0 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:03.397118 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e74d7b-ef31-49e8-944f-219310238af0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:03.397118 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e74d7b-ef31-49e8-944f-219310238af0 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:03.398425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 570/1142] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 13:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:30:03.403974 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19524a-8317-410d-bc16-07aa5d006352 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:03.405014 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19524a-8317-410d-bc16-07aa5d006352 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:03.405341 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19524a-8317-410d-bc16-07aa5d006352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:03.470243 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-7d19524a-8317-410d-bc16-07aa5d006352 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:30:03.471436 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-7d19524a-8317-410d-bc16-07aa5d006352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:03.489877 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:30:03.568308 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 573/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 13:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:30:03.858730 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e74d7b-ef31-49e8-944f-219310238af0 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:30:03.860260 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c8e74d7b-ef31-49e8-944f-219310238af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:03.961198 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e74d7b-ef31-49e8-944f-219310238af0 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:30:04.026621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:30:04.033761 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabfd21e-081a-4171-b627-9f76c7d4f381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:30:04.034362 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfd21e-081a-4171-b627-9f76c7d4f381 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:04.034628 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfd21e-081a-4171-b627-9f76c7d4f381 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:04.034899 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfd21e-081a-4171-b627-9f76c7d4f381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:04.036462 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-eabfd21e-081a-4171-b627-9f76c7d4f381 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:04.145981 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabfd21e-081a-4171-b627-9f76c7d4f381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:30:04.147332 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabfd21e-081a-4171-b627-9f76c7d4f381 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:30:04.165987 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 571/1144] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:30:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:30:06.356011 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff4ac5-ce1d-42c2-8624-4483075de78f None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:06.356644 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff4ac5-ce1d-42c2-8624-4483075de78f None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:06.356644 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff4ac5-ce1d-42c2-8624-4483075de78f None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:06.358687 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 574/1145] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 13:30:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:30:06.364151 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70481b-5aab-49b9-9dd1-14c93936a3b4 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:06.364651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70481b-5aab-49b9-9dd1-14c93936a3b4 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:06.364748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70481b-5aab-49b9-9dd1-14c93936a3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:06.431144 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8c70481b-5aab-49b9-9dd1-14c93936a3b4 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:30:06.432336 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8c70481b-5aab-49b9-9dd1-14c93936a3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:06.480321 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:30:06.531149 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 572/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 13:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:30:06.817680 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff4ac5-ce1d-42c2-8624-4483075de78f None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:30:06.819212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-6dff4ac5-ce1d-42c2-8624-4483075de78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:06.922275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff4ac5-ce1d-42c2-8624-4483075de78f None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:30:06.970295 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:30:06.995327 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8dd518-5132-4b75-a9f0-3c645078baf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:30:06.995930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8dd518-5132-4b75-a9f0-3c645078baf8 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:06.996210 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8dd518-5132-4b75-a9f0-3c645078baf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:06.996483 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8dd518-5132-4b75-a9f0-3c645078baf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:06.998661 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-ed8dd518-5132-4b75-a9f0-3c645078baf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:07.105463 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8dd518-5132-4b75-a9f0-3c645078baf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:30:07.106803 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8dd518-5132-4b75-a9f0-3c645078baf8 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:30:07.126784 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 575/1147] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:30:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:30:16.529115 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=22 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:30:17.124759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=66 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:30:27.177382 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42499a3-2a3c-4246-beac-1611929cbf44 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:27.177928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42499a3-2a3c-4246-beac-1611929cbf44 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:27.177928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42499a3-2a3c-4246-beac-1611929cbf44 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:27.179165 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 573/1148] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:30:27.184736 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6abc44-ccfa-43b3-97a9-bf6bb6ed7ac2 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:27.185236 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6abc44-ccfa-43b3-97a9-bf6bb6ed7ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:27.185340 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6abc44-ccfa-43b3-97a9-bf6bb6ed7ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:27.255377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-9f6abc44-ccfa-43b3-97a9-bf6bb6ed7ac2 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:30:27.256606 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-9f6abc44-ccfa-43b3-97a9-bf6bb6ed7ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:27.351578 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 576/1149] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:30:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:30:27.652017 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b731889-e974-4719-8870-ed7ac37ad4bd None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:27.652395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b731889-e974-4719-8870-ed7ac37ad4bd None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:27.652624 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b731889-e974-4719-8870-ed7ac37ad4bd None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:27.654181 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 574/1150] 10.222.0.23 () {54 vars in 962 bytes} [Thu Oct 20 13:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:30:27.659368 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18cb08-f4e6-4deb-b173-3362fb378f10 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:27.659729 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18cb08-f4e6-4deb-b173-3362fb378f10 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:27.660014 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18cb08-f4e6-4deb-b173-3362fb378f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:27.736153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-ce18cb08-f4e6-4deb-b173-3362fb378f10 None None] MFA Rules not processed for user `87b5093e963c4753928a8c6091afc52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:30:27.737294 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-ce18cb08-f4e6-4deb-b173-3362fb378f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:27.757403 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:30:27.814461 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 577/1151] 10.222.0.23 () {58 vars in 1059 bytes} [Thu Oct 20 13:30:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:30:27.822233 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b731889-e974-4719-8870-ed7ac37ad4bd None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:30:27.823760 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2b731889-e974-4719-8870-ed7ac37ad4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:27.930145 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b731889-e974-4719-8870-ed7ac37ad4bd None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:30:28.002754 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75523f-4a4a-45e3-8288-554ede499bce service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87b5093e963c4753928a8c6091afc52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:30:28.003272 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75523f-4a4a-45e3-8288-554ede499bce service glance] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:28.003546 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75523f-4a4a-45e3-8288-554ede499bce service glance] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:28.003810 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75523f-4a4a-45e3-8288-554ede499bce service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:28.005414 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ce75523f-4a4a-45e3-8288-554ede499bce service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:28.057741 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:30:28.116087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75523f-4a4a-45e3-8288-554ede499bce service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:30:28.117556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75523f-4a4a-45e3-8288-554ede499bce service glance] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:30:28.136510 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 575/1152] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:30:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:30:30.385782 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdec93b-a05b-4795-9c21-ff6ffa6b3bfd None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:30.386377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdec93b-a05b-4795-9c21-ff6ffa6b3bfd None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:30.386377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdec93b-a05b-4795-9c21-ff6ffa6b3bfd None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:30.387686 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 578/1153] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:30:30.393727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ce2e2-2e08-4253-8524-12b10b26ea6b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:30.394158 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ce2e2-2e08-4253-8524-12b10b26ea6b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:30.394254 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ce2e2-2e08-4253-8524-12b10b26ea6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:30.459286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1b8ce2e2-2e08-4253-8524-12b10b26ea6b None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:30:30.460475 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1b8ce2e2-2e08-4253-8524-12b10b26ea6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:30.557708 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 576/1154] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:30:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:30:30.839275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdec93b-a05b-4795-9c21-ff6ffa6b3bfd None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:30:30.840817 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-1bdec93b-a05b-4795-9c21-ff6ffa6b3bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:30.962631 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdec93b-a05b-4795-9c21-ff6ffa6b3bfd None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:30:31.011925 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:30:31.055370 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f577dcf-8644-4104-b230-700cdb498fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:30:31.055974 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f577dcf-8644-4104-b230-700cdb498fad admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:31.056247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f577dcf-8644-4104-b230-700cdb498fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:31.056514 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f577dcf-8644-4104-b230-700cdb498fad admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:31.057182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f577dcf-8644-4104-b230-700cdb498fad admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:30:31.058292 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: WARNING py.warnings [None req-0f577dcf-8644-4104-b230-700cdb498fad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 13:30:31.058292 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: warnings.warn(msg) Oct 20 13:30:31.058292 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: Oct 20 13:30:31.059422 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f577dcf-8644-4104-b230-700cdb498fad admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:30:31.078974 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 579/1155] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 20 13:30:30 2022] GET /identity/v3/projects => generated 2553 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:30:33.266699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4979b-53fc-4ffb-a51a-32cff8f12466 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:33.267250 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4979b-53fc-4ffb-a51a-32cff8f12466 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:33.267250 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4979b-53fc-4ffb-a51a-32cff8f12466 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:33.268568 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 577/1156] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:30:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:30:33.274229 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ceb87-441a-412c-ac89-e83a09e1e3ca None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:33.274709 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ceb87-441a-412c-ac89-e83a09e1e3ca None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:33.274825 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ceb87-441a-412c-ac89-e83a09e1e3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:33.343365 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-c25ceb87-441a-412c-ac89-e83a09e1e3ca None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:30:33.344561 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-c25ceb87-441a-412c-ac89-e83a09e1e3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:33.448253 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 580/1157] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:30:33.743187 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4979b-53fc-4ffb-a51a-32cff8f12466 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:30:33.744716 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-90c4979b-53fc-4ffb-a51a-32cff8f12466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:33.755946 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:30:33.902368 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4979b-53fc-4ffb-a51a-32cff8f12466 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:30:33.985968 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:30:34.019627 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c8527-e501-4a08-a44c-dc15b2b1d998 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:30:34.020263 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c8527-e501-4a08-a44c-dc15b2b1d998 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:34.020538 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c8527-e501-4a08-a44c-dc15b2b1d998 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:34.020809 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c8527-e501-4a08-a44c-dc15b2b1d998 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:34.022424 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-143c8527-e501-4a08-a44c-dc15b2b1d998 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:34.115801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c8527-e501-4a08-a44c-dc15b2b1d998 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:30:34.117444 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c8527-e501-4a08-a44c-dc15b2b1d998 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:30:34.137209 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 578/1158] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:30:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:30:36.526531 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd39bbd-5f4d-4cac-8cea-fc4bd6439dba None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:36.527049 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd39bbd-5f4d-4cac-8cea-fc4bd6439dba None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:36.527154 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd39bbd-5f4d-4cac-8cea-fc4bd6439dba None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:36.528466 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 581/1159] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:30:36.533944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930f6f3-3c54-4a41-b43f-54864b0577c6 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:36.534391 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930f6f3-3c54-4a41-b43f-54864b0577c6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:36.534496 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930f6f3-3c54-4a41-b43f-54864b0577c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:36.602432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a930f6f3-3c54-4a41-b43f-54864b0577c6 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:30:36.603643 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a930f6f3-3c54-4a41-b43f-54864b0577c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:36.654224 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:30:36.707432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 579/1160] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:30:36.993275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd39bbd-5f4d-4cac-8cea-fc4bd6439dba None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:30:36.994825 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-8dd39bbd-5f4d-4cac-8cea-fc4bd6439dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:37.020488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:30:37.105414 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd39bbd-5f4d-4cac-8cea-fc4bd6439dba None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:30:37.187261 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce2d08-ea48-430e-bd39-43000614b7aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:30:37.187876 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce2d08-ea48-430e-bd39-43000614b7aa service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:37.188153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce2d08-ea48-430e-bd39-43000614b7aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:37.188424 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce2d08-ea48-430e-bd39-43000614b7aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:37.190091 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-65ce2d08-ea48-430e-bd39-43000614b7aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:37.270794 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:30:37.306676 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce2d08-ea48-430e-bd39-43000614b7aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:30:37.308117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce2d08-ea48-430e-bd39-43000614b7aa service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:30:37.327675 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 582/1161] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:30:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:30:39.560357 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e29565-296a-40b3-a717-af394c8b2200 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:39.560913 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e29565-296a-40b3-a717-af394c8b2200 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:39.561011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e29565-296a-40b3-a717-af394c8b2200 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:39.562965 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 580/1162] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:30:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:30:39.568555 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa470f-baab-4326-9b49-0b32312598b4 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:39.569037 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa470f-baab-4326-9b49-0b32312598b4 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:39.569122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa470f-baab-4326-9b49-0b32312598b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:39.634659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-1caa470f-baab-4326-9b49-0b32312598b4 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:30:39.635872 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-1caa470f-baab-4326-9b49-0b32312598b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:39.731523 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 583/1163] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:30:40.025002 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e29565-296a-40b3-a717-af394c8b2200 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:30:40.026546 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b9e29565-296a-40b3-a717-af394c8b2200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:40.129562 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e29565-296a-40b3-a717-af394c8b2200 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:30:40.207060 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:30:40.208725 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367cf68-cb53-4fc6-b734-50c7dc19e2cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:30:40.209687 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367cf68-cb53-4fc6-b734-50c7dc19e2cf service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:40.210040 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367cf68-cb53-4fc6-b734-50c7dc19e2cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:40.210457 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367cf68-cb53-4fc6-b734-50c7dc19e2cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:40.212558 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c367cf68-cb53-4fc6-b734-50c7dc19e2cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:40.328639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367cf68-cb53-4fc6-b734-50c7dc19e2cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:30:40.330107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367cf68-cb53-4fc6-b734-50c7dc19e2cf service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:30:40.349661 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 581/1164] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:30:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:30:42.576607 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6cc6c-a7ae-490e-9600-8a68c829803d None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:42.577201 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6cc6c-a7ae-490e-9600-8a68c829803d None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:42.577299 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6cc6c-a7ae-490e-9600-8a68c829803d None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:42.578635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 584/1165] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:30:42.583809 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed4820-2ec3-4791-87ea-16448420a7e4 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:42.584253 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed4820-2ec3-4791-87ea-16448420a7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:42.584357 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed4820-2ec3-4791-87ea-16448420a7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:42.650078 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-86ed4820-2ec3-4791-87ea-16448420a7e4 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:30:42.651291 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-86ed4820-2ec3-4791-87ea-16448420a7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:42.704588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:30:42.748639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 582/1166] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:30:43.040459 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6cc6c-a7ae-490e-9600-8a68c829803d None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:30:43.042023 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-17c6cc6c-a7ae-490e-9600-8a68c829803d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:43.078009 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:30:43.146771 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6cc6c-a7ae-490e-9600-8a68c829803d None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:30:43.220039 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb2a39-69bf-4001-91c6-970a0c208a9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:30:43.220640 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb2a39-69bf-4001-91c6-970a0c208a9d service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:43.220950 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb2a39-69bf-4001-91c6-970a0c208a9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:43.221263 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb2a39-69bf-4001-91c6-970a0c208a9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:43.222876 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-19cb2a39-69bf-4001-91c6-970a0c208a9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:43.303689 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:30:43.332606 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb2a39-69bf-4001-91c6-970a0c208a9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:30:43.334084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb2a39-69bf-4001-91c6-970a0c208a9d service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:30:43.352353 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 585/1167] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:30:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:30:45.696207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa4784-3c5e-4457-ac4c-0f18840f6141 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:45.696740 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa4784-3c5e-4457-ac4c-0f18840f6141 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:45.696740 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa4784-3c5e-4457-ac4c-0f18840f6141 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:45.698141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 583/1168] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 20 13:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:30:45.703645 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc310b-280b-43e5-9196-471f577632d7 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:45.704070 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc310b-280b-43e5-9196-471f577632d7 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:45.704191 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc310b-280b-43e5-9196-471f577632d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:45.769898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-ebfc310b-280b-43e5-9196-471f577632d7 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:30:45.771000 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-ebfc310b-280b-43e5-9196-471f577632d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:45.879624 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 586/1169] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 20 13:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:30:46.193588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa4784-3c5e-4457-ac4c-0f18840f6141 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:30:46.195266 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-caaa4784-3c5e-4457-ac4c-0f18840f6141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:46.316167 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa4784-3c5e-4457-ac4c-0f18840f6141 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:30:46.408485 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a7b1a-0b68-4caf-b339-fb9e82f39cd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:30:46.409148 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a7b1a-0b68-4caf-b339-fb9e82f39cd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:30:46.409392 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a7b1a-0b68-4caf-b339-fb9e82f39cd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:30:46.409668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a7b1a-0b68-4caf-b339-fb9e82f39cd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:30:46.411269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-323a7b1a-0b68-4caf-b339-fb9e82f39cd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:30:46.422092 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:30:46.544956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a7b1a-0b68-4caf-b339-fb9e82f39cd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:30:46.547207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a7b1a-0b68-4caf-b339-fb9e82f39cd1 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:30:46.565639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 584/1170] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:30:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:30:55.878067 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=82 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:30:56.563309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=50 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:32:02.499002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1b03f-ced9-46b8-aa41-48a6c58c0765 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:32:02.499620 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1b03f-ced9-46b8-aa41-48a6c58c0765 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:32:02.499620 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1b03f-ced9-46b8-aa41-48a6c58c0765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:32:02.574171 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-3cc1b03f-ced9-46b8-aa41-48a6c58c0765 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:32:02.575365 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-3cc1b03f-ced9-46b8-aa41-48a6c58c0765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:32:02.669736 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 587/1171] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:32:02.677732 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482d0ad-22ee-4d1f-8e0f-c25a2b276d43 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:32:02.678273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482d0ad-22ee-4d1f-8e0f-c25a2b276d43 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:32:02.678273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482d0ad-22ee-4d1f-8e0f-c25a2b276d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:32:02.758040 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e482d0ad-22ee-4d1f-8e0f-c25a2b276d43 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:32:02.759169 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e482d0ad-22ee-4d1f-8e0f-c25a2b276d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:32:02.854306 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 585/1172] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:32:02.859074 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1b03f-ced9-46b8-aa41-48a6c58c0765 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:32:02.860272 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-3cc1b03f-ced9-46b8-aa41-48a6c58c0765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:32:02.871578 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1b03f-ced9-46b8-aa41-48a6c58c0765 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:32:02.933309 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:32:02.968882 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f276135-86c0-424c-b72a-d1d0d83dfeb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:32:02.969490 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f276135-86c0-424c-b72a-d1d0d83dfeb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:32:02.969772 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f276135-86c0-424c-b72a-d1d0d83dfeb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:32:02.970045 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f276135-86c0-424c-b72a-d1d0d83dfeb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:32:02.970686 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f276135-86c0-424c-b72a-d1d0d83dfeb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:32:02.970988 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f276135-86c0-424c-b72a-d1d0d83dfeb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:32:02.972008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: WARNING py.warnings [None req-8f276135-86c0-424c-b72a-d1d0d83dfeb5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 20 13:32:02.972008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: warnings.warn(msg) Oct 20 13:32:02.972008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: Oct 20 13:32:02.972914 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f276135-86c0-424c-b72a-d1d0d83dfeb5 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:32:02.983925 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 588/1173] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:32:02.993477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e482d0ad-22ee-4d1f-8e0f-c25a2b276d43 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:32:03.003487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e482d0ad-22ee-4d1f-8e0f-c25a2b276d43 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:32:03.064342 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:32:03.098284 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7a0bb-5fa6-4d61-8ef6-f3364e9d3f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:32:03.098867 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7a0bb-5fa6-4d61-8ef6-f3364e9d3f9a admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:32:03.099149 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7a0bb-5fa6-4d61-8ef6-f3364e9d3f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:32:03.099425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7a0bb-5fa6-4d61-8ef6-f3364e9d3f9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:32:03.100062 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7a0bb-5fa6-4d61-8ef6-f3364e9d3f9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:32:03.101114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: WARNING py.warnings [None req-56d7a0bb-5fa6-4d61-8ef6-f3364e9d3f9a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 13:32:03.101114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: warnings.warn(msg) Oct 20 13:32:03.101114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: Oct 20 13:32:03.102303 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7a0bb-5fa6-4d61-8ef6-f3364e9d3f9a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:32:03.158291 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 586/1174] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:32:02 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:32:03.162482 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f276135-86c0-424c-b72a-d1d0d83dfeb5 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:32:03.164037 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-8f276135-86c0-424c-b72a-d1d0d83dfeb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:32:03.197668 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f276135-86c0-424c-b72a-d1d0d83dfeb5 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:32:03.294790 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01c3c8-8783-4f8c-b4f8-881287911518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:32:03.296110 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01c3c8-8783-4f8c-b4f8-881287911518 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:32:03.296397 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01c3c8-8783-4f8c-b4f8-881287911518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:32:03.296672 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01c3c8-8783-4f8c-b4f8-881287911518 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:32:03.297348 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e01c3c8-8783-4f8c-b4f8-881287911518 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:32:03.298366 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: WARNING py.warnings [None req-9e01c3c8-8783-4f8c-b4f8-881287911518 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 13:32:03.298366 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: warnings.warn(msg) Oct 20 13:32:03.298366 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: Oct 20 13:32:03.299580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e01c3c8-8783-4f8c-b4f8-881287911518 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:32:03.341577 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 589/1175] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:32:03 2022] POST /identity/v3/users => generated 290 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:32:03.346104 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7a0bb-5fa6-4d61-8ef6-f3364e9d3f9a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:32:03.347689 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-56d7a0bb-5fa6-4d61-8ef6-f3364e9d3f9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:32:03.409917 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7a0bb-5fa6-4d61-8ef6-f3364e9d3f9a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:32:03.492907 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:32:03.505062 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce9952-f22e-4b18-9a94-b1cdfba0f363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:32:03.505681 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce9952-f22e-4b18-9a94-b1cdfba0f363 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:32:03.505960 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce9952-f22e-4b18-9a94-b1cdfba0f363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:32:03.506233 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce9952-f22e-4b18-9a94-b1cdfba0f363 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:32:03.506885 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce9952-f22e-4b18-9a94-b1cdfba0f363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:32:03.508499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce9952-f22e-4b18-9a94-b1cdfba0f363 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:32:03.522041 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 587/1176] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:32:03.526123 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01c3c8-8783-4f8c-b4f8-881287911518 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:32:03.527630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-9e01c3c8-8783-4f8c-b4f8-881287911518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:32:03.539771 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01c3c8-8783-4f8c-b4f8-881287911518 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:32:03.565177 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:32:03.634713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c20be3-8abe-4bb4-9d8d-2e11d564602e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:32:03.635267 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c20be3-8abe-4bb4-9d8d-2e11d564602e admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:32:03.635541 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c20be3-8abe-4bb4-9d8d-2e11d564602e admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:32:03.635806 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c20be3-8abe-4bb4-9d8d-2e11d564602e admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:32:03.636436 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c20be3-8abe-4bb4-9d8d-2e11d564602e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:32:03.638053 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c20be3-8abe-4bb4-9d8d-2e11d564602e admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:32:03.650488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 590/1177] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:32:03.655427 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce9952-f22e-4b18-9a94-b1cdfba0f363 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:32:03.656971 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-19ce9952-f22e-4b18-9a94-b1cdfba0f363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:32:03.690149 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce9952-f22e-4b18-9a94-b1cdfba0f363 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:32:03.782419 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f029d-468d-4bc1-9b42-241132dbfe27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:32:03.783069 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f029d-468d-4bc1-9b42-241132dbfe27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:32:03.783347 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f029d-468d-4bc1-9b42-241132dbfe27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:32:03.783620 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f029d-468d-4bc1-9b42-241132dbfe27 admin admin] PATH_INFO: `/v3/projects/43fac589b5bb4ca48abbfabb65b0ee55/users/06d5b50eff464de8b67a50db106612c9/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:32:03.805769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f029d-468d-4bc1-9b42-241132dbfe27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43fac589b5bb4ca48abbfabb65b0ee55, user_id=06d5b50eff464de8b67a50db106612c9, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:32:03.808399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f029d-468d-4bc1-9b42-241132dbfe27 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:32:03.834040 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 588/1178] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:32:03 2022] PUT /identity/v3/projects/43fac589b5bb4ca48abbfabb65b0ee55/users/06d5b50eff464de8b67a50db106612c9/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:32:03.844991 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02620b-42e7-4796-8bb3-a20c58b47569 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:32:03.845450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02620b-42e7-4796-8bb3-a20c58b47569 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:32:03.845738 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02620b-42e7-4796-8bb3-a20c58b47569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:32:03.902982 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-3a02620b-42e7-4796-8bb3-a20c58b47569 None None] MFA Rules not processed for user `06d5b50eff464de8b67a50db106612c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:32:03.904104 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-3a02620b-42e7-4796-8bb3-a20c58b47569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:32:03.979363 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 591/1179] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:32:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:32:03.992670 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24450019-0d28-45e2-bd9f-1ce6edd45989 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:32:03.992992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24450019-0d28-45e2-bd9f-1ce6edd45989 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:32:03.993269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24450019-0d28-45e2-bd9f-1ce6edd45989 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:32:03.994592 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 589/1180] 10.222.0.23 () {54 vars in 963 bytes} [Thu Oct 20 13:32:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:32:04.000004 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b32bf-6b25-428d-b3d2-eac0fba8291d None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:32:04.000287 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b32bf-6b25-428d-b3d2-eac0fba8291d None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:32:04.000559 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b32bf-6b25-428d-b3d2-eac0fba8291d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:32:04.032152 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:32:04.077620 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-af4b32bf-6b25-428d-b3d2-eac0fba8291d None None] MFA Rules not processed for user `349ba417c3fc49ecaa11a65066dc54f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:32:04.078798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-af4b32bf-6b25-428d-b3d2-eac0fba8291d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:32:04.153319 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 592/1181] 10.222.0.23 () {58 vars in 1060 bytes} [Thu Oct 20 13:32:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:32:04.161160 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24450019-0d28-45e2-bd9f-1ce6edd45989 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:32:04.162660 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-24450019-0d28-45e2-bd9f-1ce6edd45989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:32:04.212739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:32:04.267354 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24450019-0d28-45e2-bd9f-1ce6edd45989 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:32:04.340332 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9736886-1f5e-4a1f-b7aa-1d888ae999c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:32:04.340953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9736886-1f5e-4a1f-b7aa-1d888ae999c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:32:04.341234 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9736886-1f5e-4a1f-b7aa-1d888ae999c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:32:04.341503 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9736886-1f5e-4a1f-b7aa-1d888ae999c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:32:04.343126 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f9736886-1f5e-4a1f-b7aa-1d888ae999c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:32:04.441033 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9736886-1f5e-4a1f-b7aa-1d888ae999c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:32:04.442443 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9736886-1f5e-4a1f-b7aa-1d888ae999c7 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:32:04.445877 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:32:04.461010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 590/1182] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:32:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:32:04.507888 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b56d6-fd3a-48ae-9fb7-b51bc097088c None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:32:04.508184 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b56d6-fd3a-48ae-9fb7-b51bc097088c None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:32:04.508466 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b56d6-fd3a-48ae-9fb7-b51bc097088c None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:32:04.510008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 593/1183] 10.222.0.23 () {52 vars in 830 bytes} [Thu Oct 20 13:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:32:04.513513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9736886-1f5e-4a1f-b7aa-1d888ae999c7 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:32:04.514771 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f9736886-1f5e-4a1f-b7aa-1d888ae999c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:32:04.612729 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9736886-1f5e-4a1f-b7aa-1d888ae999c7 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:32:04.706171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091b9afd-ed08-4ffb-b9e3-228d3d019b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:32:04.706826 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b9afd-ed08-4ffb-b9e3-228d3d019b36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:32:04.706988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b9afd-ed08-4ffb-b9e3-228d3d019b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:32:04.707260 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b9afd-ed08-4ffb-b9e3-228d3d019b36 admin admin] PATH_INFO: `/v3/users/06d5b50eff464de8b67a50db106612c9` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:32:04.720838 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091b9afd-ed08-4ffb-b9e3-228d3d019b36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06d5b50eff464de8b67a50db106612c9)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:32:04.721883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: WARNING py.warnings [None req-091b9afd-ed08-4ffb-b9e3-228d3d019b36 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 13:32:04.721883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: warnings.warn(msg) Oct 20 13:32:04.721883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: Oct 20 13:32:04.723085 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091b9afd-ed08-4ffb-b9e3-228d3d019b36 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:32:04.733212 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:32:04.790587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-091b9afd-ed08-4ffb-b9e3-228d3d019b36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06d5b50eff464de8b67a50db106612c9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:32:04.791037 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-091b9afd-ed08-4ffb-b9e3-228d3d019b36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06d5b50eff464de8b67a50db106612c9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:32:04.805294 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-091b9afd-ed08-4ffb-b9e3-228d3d019b36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06d5b50eff464de8b67a50db106612c9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:32:04.814846 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-091b9afd-ed08-4ffb-b9e3-228d3d019b36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06d5b50eff464de8b67a50db106612c9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:32:04.815448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-091b9afd-ed08-4ffb-b9e3-228d3d019b36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06d5b50eff464de8b67a50db106612c9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:32:04.844453 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 591/1184] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:32:04 2022] DELETE /identity/v3/users/06d5b50eff464de8b67a50db106612c9 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:32:04.848825 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45b904-6c19-4cd0-b9b6-2cc4bf13b6ba None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:32:04.849303 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45b904-6c19-4cd0-b9b6-2cc4bf13b6ba None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:32:04.849418 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45b904-6c19-4cd0-b9b6-2cc4bf13b6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:32:04.906522 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-8e45b904-6c19-4cd0-b9b6-2cc4bf13b6ba None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:32:04.907800 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-8e45b904-6c19-4cd0-b9b6-2cc4bf13b6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:32:04.981355 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:32:04.994200 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 594/1185] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:32:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:32:05.005007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091b9afd-ed08-4ffb-b9e3-228d3d019b36 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:32:05.006218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-091b9afd-ed08-4ffb-b9e3-228d3d019b36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:32:05.104305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091b9afd-ed08-4ffb-b9e3-228d3d019b36 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:32:05.159557 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:32:05.180392 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5686a40-33ab-4344-afca-605efa749b54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:32:05.181043 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5686a40-33ab-4344-afca-605efa749b54 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:32:05.181322 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5686a40-33ab-4344-afca-605efa749b54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:32:05.181593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5686a40-33ab-4344-afca-605efa749b54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:32:05.183170 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b5686a40-33ab-4344-afca-605efa749b54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:32:05.291851 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5686a40-33ab-4344-afca-605efa749b54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:32:05.293243 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5686a40-33ab-4344-afca-605efa749b54 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:32:05.312028 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 592/1186] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:32:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:32:05.672158 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45b904-6c19-4cd0-b9b6-2cc4bf13b6ba None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:32:05.673427 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-8e45b904-6c19-4cd0-b9b6-2cc4bf13b6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:32:05.684519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45b904-6c19-4cd0-b9b6-2cc4bf13b6ba None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:32:05.782156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afab575-76a3-4a76-8f16-b6b377678182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:32:05.782783 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afab575-76a3-4a76-8f16-b6b377678182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:32:05.783058 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afab575-76a3-4a76-8f16-b6b377678182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:32:05.783329 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afab575-76a3-4a76-8f16-b6b377678182 admin admin] PATH_INFO: `/v3/projects/43fac589b5bb4ca48abbfabb65b0ee55` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:32:05.793042 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afab575-76a3-4a76-8f16-b6b377678182 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43fac589b5bb4ca48abbfabb65b0ee55)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:32:05.794076 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: WARNING py.warnings [None req-6afab575-76a3-4a76-8f16-b6b377678182 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 13:32:05.794076 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: warnings.warn(msg) Oct 20 13:32:05.794076 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: Oct 20 13:32:05.795648 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afab575-76a3-4a76-8f16-b6b377678182 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:32:05.851767 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6afab575-76a3-4a76-8f16-b6b377678182 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43fac589b5bb4ca48abbfabb65b0ee55', 'request_id': 'req-6afab575-76a3-4a76-8f16-b6b377678182'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:32:05.864837 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6afab575-76a3-4a76-8f16-b6b377678182 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43fac589b5bb4ca48abbfabb65b0ee55', 'request_id': 'req-6afab575-76a3-4a76-8f16-b6b377678182'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:32:05.873169 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6afab575-76a3-4a76-8f16-b6b377678182 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43fac589b5bb4ca48abbfabb65b0ee55', 'request_id': 'req-6afab575-76a3-4a76-8f16-b6b377678182'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:32:05.897694 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6afab575-76a3-4a76-8f16-b6b377678182 admin admin] The token cache is being invalidate because project 43fac589b5bb4ca48abbfabb65b0ee55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:32:05.897982 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6afab575-76a3-4a76-8f16-b6b377678182 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:32:05.898404 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6afab575-76a3-4a76-8f16-b6b377678182 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:32:05.899765 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 595/1187] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:32:05 2022] DELETE /identity/v3/projects/43fac589b5bb4ca48abbfabb65b0ee55 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:32:15.309471 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=62 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:32:15.895619 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:33:30.708756 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a91f4-4c50-4733-9a32-86c90357fe2c None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:30.709379 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a91f4-4c50-4733-9a32-86c90357fe2c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:30.709379 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a91f4-4c50-4733-9a32-86c90357fe2c None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:30.710670 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 593/1188] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 13:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:33:30.715618 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d976f-cc7e-451e-8e2c-88229438b995 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:30.716051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d976f-cc7e-451e-8e2c-88229438b995 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:30.716162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d976f-cc7e-451e-8e2c-88229438b995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:30.796358 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-251d976f-cc7e-451e-8e2c-88229438b995 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:33:30.797518 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-251d976f-cc7e-451e-8e2c-88229438b995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:33:30.899509 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 596/1189] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 13:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:33:31.054820 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a91f4-4c50-4733-9a32-86c90357fe2c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:33:31.056932 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-340a91f4-4c50-4733-9a32-86c90357fe2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:33:31.174383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a91f4-4c50-4733-9a32-86c90357fe2c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:33:31.179789 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c99b723-a398-423e-a985-b9f602c20963 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:33:31.180382 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99b723-a398-423e-a985-b9f602c20963 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:31.180662 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99b723-a398-423e-a985-b9f602c20963 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:31.180962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99b723-a398-423e-a985-b9f602c20963 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:31.188771 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c99b723-a398-423e-a985-b9f602c20963 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:33:31.190605 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c99b723-a398-423e-a985-b9f602c20963 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:33:31.199701 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 594/1190] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:33:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:33:32.292195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b2d95-9669-4aaa-92f7-9deb07cd9c9d None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:32.292741 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b2d95-9669-4aaa-92f7-9deb07cd9c9d None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:32.292741 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b2d95-9669-4aaa-92f7-9deb07cd9c9d None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:32.294128 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 597/1191] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 13:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:33:32.301368 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3bcec-14f4-4412-864f-c00345ee2a63 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:32.301872 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3bcec-14f4-4412-864f-c00345ee2a63 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:32.301982 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3bcec-14f4-4412-864f-c00345ee2a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:32.322237 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-56a3bcec-14f4-4412-864f-c00345ee2a63 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:33:32.323511 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-56a3bcec-14f4-4412-864f-c00345ee2a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:33:32.331577 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 595/1192] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 13:33:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:33:32.484700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b2d95-9669-4aaa-92f7-9deb07cd9c9d None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:33:32.500004 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b2d95-9669-4aaa-92f7-9deb07cd9c9d None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:33:32.505791 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1026a36-f7f8-4e60-9721-2a79efec583c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:33:32.507047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1026a36-f7f8-4e60-9721-2a79efec583c service nova] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:32.507325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1026a36-f7f8-4e60-9721-2a79efec583c service nova] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:32.507604 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1026a36-f7f8-4e60-9721-2a79efec583c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:32.515887 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1026a36-f7f8-4e60-9721-2a79efec583c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:33:32.518251 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1026a36-f7f8-4e60-9721-2a79efec583c service nova] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:33:32.527457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 598/1193] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:33:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:33:32.944666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573cbbe-d24e-469c-a33e-535df07cd584 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:32.944991 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573cbbe-d24e-469c-a33e-535df07cd584 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:32.945272 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573cbbe-d24e-469c-a33e-535df07cd584 None None] PATH_INFO: `/v3` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:32.946616 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 596/1194] 10.222.0.20 () {54 vars in 925 bytes} [Thu Oct 20 13:33:32 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:33:32.966652 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ec14c-3f47-422f-ad15-c6d0377d5c5d None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:32.966944 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ec14c-3f47-422f-ad15-c6d0377d5c5d None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:32.967223 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ec14c-3f47-422f-ad15-c6d0377d5c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:33.026244 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-915ec14c-3f47-422f-ad15-c6d0377d5c5d None None] MFA Rules not processed for user `2440594934b64621a01c441133778155`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:33:33.027428 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-915ec14c-3f47-422f-ad15-c6d0377d5c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:33:33.043887 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:33:33.098685 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 599/1195] 10.222.0.20 () {58 vars in 1015 bytes} [Thu Oct 20 13:33:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:33:36.633831 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f722c-928b-44c8-8343-8ca7b6ef9605 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:36.634308 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f722c-928b-44c8-8343-8ca7b6ef9605 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:36.634418 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f722c-928b-44c8-8343-8ca7b6ef9605 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:36.635735 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 597/1196] 10.222.0.23 () {54 vars in 961 bytes} [Thu Oct 20 13:33:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:33:36.640638 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269e53f-81a1-4806-a0a5-9a69478c5ecb None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:36.641085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269e53f-81a1-4806-a0a5-9a69478c5ecb None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:36.641225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f269e53f-81a1-4806-a0a5-9a69478c5ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:36.661662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-f269e53f-81a1-4806-a0a5-9a69478c5ecb None None] MFA Rules not processed for user `2440594934b64621a01c441133778155`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:33:36.662872 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-f269e53f-81a1-4806-a0a5-9a69478c5ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:33:36.671556 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 600/1197] 10.222.0.23 () {58 vars in 1058 bytes} [Thu Oct 20 13:33:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:33:36.681043 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f722c-928b-44c8-8343-8ca7b6ef9605 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:33:36.695475 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f722c-928b-44c8-8343-8ca7b6ef9605 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:33:36.700349 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a5071f-340a-49c9-a191-99a66c030510 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2440594934b64621a01c441133778155', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:33:36.700974 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5071f-340a-49c9-a191-99a66c030510 service placement] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:36.701261 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5071f-340a-49c9-a191-99a66c030510 service placement] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:36.701533 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5071f-340a-49c9-a191-99a66c030510 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:36.709930 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a5071f-340a-49c9-a191-99a66c030510 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:33:36.711506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a5071f-340a-49c9-a191-99a66c030510 service placement] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:33:36.721487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 598/1198] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:33:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:33:44.006878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21e2bd-391e-4606-8dc9-92abcbd5a10c None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:44.007484 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21e2bd-391e-4606-8dc9-92abcbd5a10c None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:44.007484 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21e2bd-391e-4606-8dc9-92abcbd5a10c None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:44.008844 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 601/1199] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 13:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:33:44.016278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3ee02-dd67-416e-9a5d-e5c5e4c600d9 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:44.016728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3ee02-dd67-416e-9a5d-e5c5e4c600d9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:44.016925 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3ee02-dd67-416e-9a5d-e5c5e4c600d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:44.036245 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-dda3ee02-dd67-416e-9a5d-e5c5e4c600d9 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:33:44.037472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-dda3ee02-dd67-416e-9a5d-e5c5e4c600d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:33:44.045194 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 599/1200] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 13:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:33:44.199830 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21e2bd-391e-4606-8dc9-92abcbd5a10c None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:33:44.214088 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21e2bd-391e-4606-8dc9-92abcbd5a10c None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:33:44.219373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddacf19-8c3a-4abf-ac60-f867fef2b7f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:33:44.220570 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddacf19-8c3a-4abf-ac60-f867fef2b7f1 service nova] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:44.221057 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddacf19-8c3a-4abf-ac60-f867fef2b7f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:44.221515 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddacf19-8c3a-4abf-ac60-f867fef2b7f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:44.229742 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddacf19-8c3a-4abf-ac60-f867fef2b7f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:33:44.231329 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddacf19-8c3a-4abf-ac60-f867fef2b7f1 service nova] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:33:44.241194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 602/1201] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:33:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:33:45.303641 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27289ae4-a397-4409-968b-c575e93c2df2 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:45.304218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27289ae4-a397-4409-968b-c575e93c2df2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:45.304218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27289ae4-a397-4409-968b-c575e93c2df2 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:45.305608 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 600/1202] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 20 13:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:33:45.311871 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec776057-e033-427e-8777-a5168ac736c1 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:45.312275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec776057-e033-427e-8777-a5168ac736c1 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:45.312421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec776057-e033-427e-8777-a5168ac736c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:45.332120 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-ec776057-e033-427e-8777-a5168ac736c1 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:33:45.333348 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-ec776057-e033-427e-8777-a5168ac736c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:33:45.341136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 603/1203] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 20 13:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:33:45.493031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27289ae4-a397-4409-968b-c575e93c2df2 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:33:45.508320 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27289ae4-a397-4409-968b-c575e93c2df2 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:33:45.513709 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ffbb66-0df9-498b-a67c-273b912803ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:33:45.514303 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffbb66-0df9-498b-a67c-273b912803ee service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:45.514584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffbb66-0df9-498b-a67c-273b912803ee service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:45.514859 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffbb66-0df9-498b-a67c-273b912803ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:45.523352 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ffbb66-0df9-498b-a67c-273b912803ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:33:45.524952 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ffbb66-0df9-498b-a67c-273b912803ee service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:33:45.534314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 601/1204] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:33:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:33:49.558559 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1aa21-e25a-4637-a509-f364d3c74162 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:49.559091 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1aa21-e25a-4637-a509-f364d3c74162 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:49.559091 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1aa21-e25a-4637-a509-f364d3c74162 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:49.560465 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 604/1205] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 13:33:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:33:49.566717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2333689-ad80-4b21-b567-28f254f9747d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:49.567112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2333689-ad80-4b21-b567-28f254f9747d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:49.567278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2333689-ad80-4b21-b567-28f254f9747d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:49.587006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e2333689-ad80-4b21-b567-28f254f9747d None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:33:49.588208 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e2333689-ad80-4b21-b567-28f254f9747d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:33:49.596023 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 602/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 13:33:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:33:49.906728 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1aa21-e25a-4637-a509-f364d3c74162 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:33:49.909773 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-a3d1aa21-e25a-4637-a509-f364d3c74162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:33:50.002980 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1aa21-e25a-4637-a509-f364d3c74162 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:33:50.006872 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741d255-1495-4569-9c98-8ccb48f524f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:33:50.007506 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741d255-1495-4569-9c98-8ccb48f524f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:50.008589 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741d255-1495-4569-9c98-8ccb48f524f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:50.008589 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741d255-1495-4569-9c98-8ccb48f524f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:50.018408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741d255-1495-4569-9c98-8ccb48f524f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:33:50.019851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741d255-1495-4569-9c98-8ccb48f524f4 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:33:50.029620 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 605/1207] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:33:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:33:52.213314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458244c8-dca6-4bf0-a59b-67cb835d684c None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:52.213878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458244c8-dca6-4bf0-a59b-67cb835d684c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:52.213878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458244c8-dca6-4bf0-a59b-67cb835d684c None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:52.215193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 603/1208] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 13:33:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:33:52.220452 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5174c-65d5-4f63-ba1f-61bb241f8f2e None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:52.220886 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5174c-65d5-4f63-ba1f-61bb241f8f2e None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:52.221022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5174c-65d5-4f63-ba1f-61bb241f8f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:52.241957 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-1ba5174c-65d5-4f63-ba1f-61bb241f8f2e None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:33:52.243145 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-1ba5174c-65d5-4f63-ba1f-61bb241f8f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:33:52.250833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 606/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 13:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:33:52.542732 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458244c8-dca6-4bf0-a59b-67cb835d684c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:33:52.553324 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458244c8-dca6-4bf0-a59b-67cb835d684c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:33:52.556719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca9301-7bec-4053-9d32-1f7e75924016 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:33:52.557352 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca9301-7bec-4053-9d32-1f7e75924016 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:52.557629 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca9301-7bec-4053-9d32-1f7e75924016 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:52.557897 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca9301-7bec-4053-9d32-1f7e75924016 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:52.567239 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca9301-7bec-4053-9d32-1f7e75924016 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:33:52.568674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca9301-7bec-4053-9d32-1f7e75924016 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:33:52.578117 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 604/1210] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:33:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:33:56.140456 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3756a41-cb95-48ef-8175-a21e3f955569 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:56.141028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3756a41-cb95-48ef-8175-a21e3f955569 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:56.141028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3756a41-cb95-48ef-8175-a21e3f955569 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:56.142483 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 607/1211] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 13:33:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:33:56.147669 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0b360-cb58-4f49-9d0b-b2b27f1e91a4 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:56.148071 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0b360-cb58-4f49-9d0b-b2b27f1e91a4 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:56.148225 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0b360-cb58-4f49-9d0b-b2b27f1e91a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:56.155735 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=81 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:33:56.168732 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-83c0b360-cb58-4f49-9d0b-b2b27f1e91a4 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:33:56.169956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-83c0b360-cb58-4f49-9d0b-b2b27f1e91a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:33:56.178844 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 605/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 13:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:33:56.488754 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3756a41-cb95-48ef-8175-a21e3f955569 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:33:56.490426 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-d3756a41-cb95-48ef-8175-a21e3f955569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:33:56.557249 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:33:56.570950 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3756a41-cb95-48ef-8175-a21e3f955569 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:33:56.574503 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae7aac-ed8a-4d2c-857d-40f51501738e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:33:56.575109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae7aac-ed8a-4d2c-857d-40f51501738e service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:56.575387 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae7aac-ed8a-4d2c-857d-40f51501738e service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:56.575662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae7aac-ed8a-4d2c-857d-40f51501738e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:56.585496 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae7aac-ed8a-4d2c-857d-40f51501738e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:33:56.586928 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae7aac-ed8a-4d2c-857d-40f51501738e service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:33:56.597138 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 608/1213] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:33:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:33:58.734810 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5afdbc-b1a2-41b9-a84c-c1f645bfa2d3 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:58.735372 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5afdbc-b1a2-41b9-a84c-c1f645bfa2d3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:58.735495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5afdbc-b1a2-41b9-a84c-c1f645bfa2d3 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:58.736820 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 606/1214] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 20 13:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:33:58.742091 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6f525-d7b4-41bb-a5c7-bc8fb1d17e23 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:58.742514 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6f525-d7b4-41bb-a5c7-bc8fb1d17e23 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:58.742623 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6f525-d7b4-41bb-a5c7-bc8fb1d17e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:58.762535 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-ecb6f525-d7b4-41bb-a5c7-bc8fb1d17e23 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:33:58.763642 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-ecb6f525-d7b4-41bb-a5c7-bc8fb1d17e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:33:58.771964 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 609/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 20 13:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:33:59.072539 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5afdbc-b1a2-41b9-a84c-c1f645bfa2d3 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:33:59.074234 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ae5afdbc-b1a2-41b9-a84c-c1f645bfa2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:33:59.087061 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5afdbc-b1a2-41b9-a84c-c1f645bfa2d3 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:33:59.090545 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59887aa8-9d79-4b78-b4cb-947f896bf947 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:33:59.091140 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59887aa8-9d79-4b78-b4cb-947f896bf947 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:33:59.091417 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59887aa8-9d79-4b78-b4cb-947f896bf947 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:33:59.091687 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59887aa8-9d79-4b78-b4cb-947f896bf947 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:33:59.100293 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59887aa8-9d79-4b78-b4cb-947f896bf947 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:33:59.101749 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59887aa8-9d79-4b78-b4cb-947f896bf947 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:33:59.111450 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 607/1216] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:33:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:34:08.757703 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=11 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:34:09.106784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=8 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:23.846643 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1069c1e-45c0-45e2-ad39-80fa56a54da8 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:23.847323 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1069c1e-45c0-45e2-ad39-80fa56a54da8 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:23.847323 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1069c1e-45c0-45e2-ad39-80fa56a54da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:23.868737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-b1069c1e-45c0-45e2-ad39-80fa56a54da8 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:23.869965 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b1069c1e-45c0-45e2-ad39-80fa56a54da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:23.885173 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 610/1217] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:23.896045 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116faab-205f-4ef3-8291-d626c04eabb7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:23.896693 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116faab-205f-4ef3-8291-d626c04eabb7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:23.896787 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116faab-205f-4ef3-8291-d626c04eabb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:23.923747 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d116faab-205f-4ef3-8291-d626c04eabb7 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:23.925392 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d116faab-205f-4ef3-8291-d626c04eabb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:23.926097 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34802d45-e2ca-4c69-9490-170243680154 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:23.926387 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34802d45-e2ca-4c69-9490-170243680154 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:23.926663 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34802d45-e2ca-4c69-9490-170243680154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:23.945729 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 608/1218] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:23.950886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116faab-205f-4ef3-8291-d626c04eabb7 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:23.953727 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-34802d45-e2ca-4c69-9490-170243680154 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:23.954940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-34802d45-e2ca-4c69-9490-170243680154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:23.966180 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116faab-205f-4ef3-8291-d626c04eabb7 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:23.969713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 611/1219] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:23.975953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e117d84-cbeb-446d-88b1-28fd4d384965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:23.976963 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e117d84-cbeb-446d-88b1-28fd4d384965 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:23.977481 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e117d84-cbeb-446d-88b1-28fd4d384965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:23.977983 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e117d84-cbeb-446d-88b1-28fd4d384965 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:23.978897 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e117d84-cbeb-446d-88b1-28fd4d384965 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:35:23.979022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a581357-e4bf-4e2d-87c3-a2ca37bd162d None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:23.979316 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a581357-e4bf-4e2d-87c3-a2ca37bd162d None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:23.979592 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a581357-e4bf-4e2d-87c3-a2ca37bd162d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:23.979988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e117d84-cbeb-446d-88b1-28fd4d384965 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:23.981524 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: WARNING py.warnings [None req-9e117d84-cbeb-446d-88b1-28fd4d384965 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 20 13:35:23.981524 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: warnings.warn(msg) Oct 20 13:35:23.981524 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: Oct 20 13:35:23.982895 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e117d84-cbeb-446d-88b1-28fd4d384965 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:24.002079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 609/1220] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:35:24.008018 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-5a581357-e4bf-4e2d-87c3-a2ca37bd162d None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:24.009291 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-5a581357-e4bf-4e2d-87c3-a2ca37bd162d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:24.012538 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e117d84-cbeb-446d-88b1-28fd4d384965 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:24.017764 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 612/1221] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:35:24.022508 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a581357-e4bf-4e2d-87c3-a2ca37bd162d None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:24.027835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e117d84-cbeb-446d-88b1-28fd4d384965 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:24.033122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a581357-e4bf-4e2d-87c3-a2ca37bd162d None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:24.038199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d227311-d892-4e40-b373-cfe7ba029209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:24.039040 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d227311-d892-4e40-b373-cfe7ba029209 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.039314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36397142-622e-4b6c-b03d-98b9f8bd47ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:24.039668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d227311-d892-4e40-b373-cfe7ba029209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.040106 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d227311-d892-4e40-b373-cfe7ba029209 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.040235 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36397142-622e-4b6c-b03d-98b9f8bd47ec admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.040559 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36397142-622e-4b6c-b03d-98b9f8bd47ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.044894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36397142-622e-4b6c-b03d-98b9f8bd47ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.044894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36397142-622e-4b6c-b03d-98b9f8bd47ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:35:24.044894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36397142-622e-4b6c-b03d-98b9f8bd47ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:24.044894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36397142-622e-4b6c-b03d-98b9f8bd47ec admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:24.045938 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d227311-d892-4e40-b373-cfe7ba029209 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:24.048672 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d227311-d892-4e40-b373-cfe7ba029209 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:24.058757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 613/1222] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:35:24.066857 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36397142-622e-4b6c-b03d-98b9f8bd47ec admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:24.077261 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36397142-622e-4b6c-b03d-98b9f8bd47ec admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:24.100360 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b89806-7528-445b-b4ed-bd2cfe668551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:24.106478 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b89806-7528-445b-b4ed-bd2cfe668551 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.107010 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b89806-7528-445b-b4ed-bd2cfe668551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.107485 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b89806-7528-445b-b4ed-bd2cfe668551 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.108596 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b89806-7528-445b-b4ed-bd2cfe668551 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:24.112411 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: WARNING py.warnings [None req-51b89806-7528-445b-b4ed-bd2cfe668551 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 13:35:24.112411 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: warnings.warn(msg) Oct 20 13:35:24.112411 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: Oct 20 13:35:24.116169 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 610/1223] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:35:24 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:35:24.117347 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b89806-7528-445b-b4ed-bd2cfe668551 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:24.125165 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d227311-d892-4e40-b373-cfe7ba029209 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:24.185324 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 614/1224] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:35:24 2022] POST /identity/v3/projects => generated 375 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:35:24.190953 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b89806-7528-445b-b4ed-bd2cfe668551 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:24.226469 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d227311-d892-4e40-b373-cfe7ba029209 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:24.317649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b89806-7528-445b-b4ed-bd2cfe668551 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:24.322032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b366ceb-63c2-4da3-87d1-a53ba1027369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:24.322859 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b366ceb-63c2-4da3-87d1-a53ba1027369 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.323310 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b366ceb-63c2-4da3-87d1-a53ba1027369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.323732 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b366ceb-63c2-4da3-87d1-a53ba1027369 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.324527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b366ceb-63c2-4da3-87d1-a53ba1027369 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:24.325737 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: WARNING py.warnings [None req-4b366ceb-63c2-4da3-87d1-a53ba1027369 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 13:35:24.325737 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: warnings.warn(msg) Oct 20 13:35:24.325737 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: Oct 20 13:35:24.327261 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b366ceb-63c2-4da3-87d1-a53ba1027369 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:24.335396 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6f508-2934-4090-a7d1-b4d1fcd3be16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:24.336031 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6f508-2934-4090-a7d1-b4d1fcd3be16 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.336316 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6f508-2934-4090-a7d1-b4d1fcd3be16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.336593 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6f508-2934-4090-a7d1-b4d1fcd3be16 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.337268 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6f508-2934-4090-a7d1-b4d1fcd3be16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:24.339212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6f508-2934-4090-a7d1-b4d1fcd3be16 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:24.369030 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 611/1225] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:35:24 2022] POST /identity/v3/users => generated 284 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:35:24.370945 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f7c2c-8676-4437-ad92-e83a7c223d36 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.371400 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f7c2c-8676-4437-ad92-e83a7c223d36 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.371837 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f7c2c-8676-4437-ad92-e83a7c223d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.383678 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 615/1226] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:35:24 2022] POST /identity/v3/users => generated 288 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:24.385620 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12f837-7dd5-4b0d-b338-55be4573d23f None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.385620 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12f837-7dd5-4b0d-b338-55be4573d23f None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.385866 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12f837-7dd5-4b0d-b338-55be4573d23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.400935 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-440f7c2c-8676-4437-ad92-e83a7c223d36 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:24.400935 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-440f7c2c-8676-4437-ad92-e83a7c223d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:24.418082 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-8b12f837-7dd5-4b0d-b338-55be4573d23f None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:24.419291 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-8b12f837-7dd5-4b0d-b338-55be4573d23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:24.436672 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 616/1227] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:24.436940 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 612/1228] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:24.437741 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f7c2c-8676-4437-ad92-e83a7c223d36 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:24.438168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e5b1c-71c3-4c02-b2cc-f8b43d16c654 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.438515 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e5b1c-71c3-4c02-b2cc-f8b43d16c654 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.438795 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e5b1c-71c3-4c02-b2cc-f8b43d16c654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.458101 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f7c2c-8676-4437-ad92-e83a7c223d36 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:24.463196 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d59068-ace2-43aa-8b1e-1d1d020d84c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:24.463837 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d59068-ace2-43aa-8b1e-1d1d020d84c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.464112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d59068-ace2-43aa-8b1e-1d1d020d84c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.464381 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d59068-ace2-43aa-8b1e-1d1d020d84c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.465066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d59068-ace2-43aa-8b1e-1d1d020d84c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:24.466658 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d59068-ace2-43aa-8b1e-1d1d020d84c2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:24.467477 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-668e5b1c-71c3-4c02-b2cc-f8b43d16c654 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:24.468662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-668e5b1c-71c3-4c02-b2cc-f8b43d16c654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:24.477094 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 617/1229] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:24.477837 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 613/1230] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:24.477967 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e5b1c-71c3-4c02-b2cc-f8b43d16c654 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:24.479095 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53591ff1-65eb-4aaa-a888-80fc098bd6d0 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.479444 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53591ff1-65eb-4aaa-a888-80fc098bd6d0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.479780 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53591ff1-65eb-4aaa-a888-80fc098bd6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.487362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e5b1c-71c3-4c02-b2cc-f8b43d16c654 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:24.492763 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59829f66-edeb-4911-be5a-527e511180a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:24.493392 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59829f66-edeb-4911-be5a-527e511180a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.493668 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59829f66-edeb-4911-be5a-527e511180a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.493940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59829f66-edeb-4911-be5a-527e511180a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.494605 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59829f66-edeb-4911-be5a-527e511180a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:24.496219 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59829f66-edeb-4911-be5a-527e511180a6 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:24.501340 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-53591ff1-65eb-4aaa-a888-80fc098bd6d0 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:24.503013 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-53591ff1-65eb-4aaa-a888-80fc098bd6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:24.507607 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 618/1231] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:24.511142 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5f1d8-c04b-4246-860e-e9cf4862f395 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.511142 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5f1d8-c04b-4246-860e-e9cf4862f395 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.511142 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5f1d8-c04b-4246-860e-e9cf4862f395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.514635 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 614/1232] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:24.516011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b7188-4f07-4179-be28-7b6a873822d5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.516395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b7188-4f07-4179-be28-7b6a873822d5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.516788 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b7188-4f07-4179-be28-7b6a873822d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.529369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-90a5f1d8-c04b-4246-860e-e9cf4862f395 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:24.530558 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-90a5f1d8-c04b-4246-860e-e9cf4862f395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:24.537676 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f34b7188-4f07-4179-be28-7b6a873822d5 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:24.538840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f34b7188-4f07-4179-be28-7b6a873822d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:24.539314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 619/1233] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:24.539314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8b2f0-c61b-4fdc-b2ee-8f7ab7c2ab30 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.539314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8b2f0-c61b-4fdc-b2ee-8f7ab7c2ab30 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.539314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8b2f0-c61b-4fdc-b2ee-8f7ab7c2ab30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.546876 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 615/1234] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:24.548191 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a05fad-e21c-40be-a12e-d51e952aeaae None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.548476 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a05fad-e21c-40be-a12e-d51e952aeaae None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.548745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a05fad-e21c-40be-a12e-d51e952aeaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.566976 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-f3a8b2f0-c61b-4fdc-b2ee-8f7ab7c2ab30 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:24.568188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-f3a8b2f0-c61b-4fdc-b2ee-8f7ab7c2ab30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:24.569267 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-37a05fad-e21c-40be-a12e-d51e952aeaae None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:24.570414 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-37a05fad-e21c-40be-a12e-d51e952aeaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:24.576776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 620/1235] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:24.577587 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8b2f0-c61b-4fdc-b2ee-8f7ab7c2ab30 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:24.578535 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 616/1236] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:24.579577 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e609b-6550-42ec-9cf9-2691795dd5f5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.579860 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e609b-6550-42ec-9cf9-2691795dd5f5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.580133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e609b-6550-42ec-9cf9-2691795dd5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.587465 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8b2f0-c61b-4fdc-b2ee-8f7ab7c2ab30 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:24.592040 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a080a6-600d-4a94-af17-4648a6b9a39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:24.592642 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a080a6-600d-4a94-af17-4648a6b9a39c admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.592953 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a080a6-600d-4a94-af17-4648a6b9a39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.593229 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a080a6-600d-4a94-af17-4648a6b9a39c admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.593901 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a080a6-600d-4a94-af17-4648a6b9a39c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:24.595517 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a080a6-600d-4a94-af17-4648a6b9a39c admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:24.605605 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-839e609b-6550-42ec-9cf9-2691795dd5f5 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:24.607331 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-839e609b-6550-42ec-9cf9-2691795dd5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:24.609641 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 621/1237] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:24.610245 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a080a6-600d-4a94-af17-4648a6b9a39c admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:24.616903 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 617/1238] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:24.618268 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83909d80-72f1-48a1-8ef8-85a1b0e703f9 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.618574 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83909d80-72f1-48a1-8ef8-85a1b0e703f9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.618854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83909d80-72f1-48a1-8ef8-85a1b0e703f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.624351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a080a6-600d-4a94-af17-4648a6b9a39c admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:24.629221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:24.629388 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f664320-d567-42c5-be1a-4b4dbacc3b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:24.629990 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f664320-d567-42c5-be1a-4b4dbacc3b0d admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.630276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f664320-d567-42c5-be1a-4b4dbacc3b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.630549 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f664320-d567-42c5-be1a-4b4dbacc3b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.631749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f664320-d567-42c5-be1a-4b4dbacc3b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:24.633420 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f664320-d567-42c5-be1a-4b4dbacc3b0d admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:24.647000 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 622/1239] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:24.647200 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-83909d80-72f1-48a1-8ef8-85a1b0e703f9 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:24.647324 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f664320-d567-42c5-be1a-4b4dbacc3b0d admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:24.647426 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-83909d80-72f1-48a1-8ef8-85a1b0e703f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:24.653905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 618/1240] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:24.654279 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f664320-d567-42c5-be1a-4b4dbacc3b0d admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:24.655124 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8af33-fbbe-4c87-b06a-09647f19ee6f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.655413 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8af33-fbbe-4c87-b06a-09647f19ee6f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.655688 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8af33-fbbe-4c87-b06a-09647f19ee6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.661223 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76ff0c-3ed2-4317-af92-4780508985b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:24.661814 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76ff0c-3ed2-4317-af92-4780508985b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.662858 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76ff0c-3ed2-4317-af92-4780508985b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.662858 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76ff0c-3ed2-4317-af92-4780508985b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.663588 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a76ff0c-3ed2-4317-af92-4780508985b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:35:24.663876 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a76ff0c-3ed2-4317-af92-4780508985b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:24.665507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a76ff0c-3ed2-4317-af92-4780508985b9 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:24.675251 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-6bb8af33-fbbe-4c87-b06a-09647f19ee6f None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:24.676454 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6bb8af33-fbbe-4c87-b06a-09647f19ee6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:24.679194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 623/1241] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:35:24.680013 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76ff0c-3ed2-4317-af92-4780508985b9 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:24.684719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 619/1242] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:24.686036 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e22259-7b68-4327-a266-afb74f31e864 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.686325 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e22259-7b68-4327-a266-afb74f31e864 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.686608 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e22259-7b68-4327-a266-afb74f31e864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.693194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76ff0c-3ed2-4317-af92-4780508985b9 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:24.698484 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a724c-483f-405c-be9e-75a56d1774c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:24.699085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a724c-483f-405c-be9e-75a56d1774c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.699363 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a724c-483f-405c-be9e-75a56d1774c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.699634 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a724c-483f-405c-be9e-75a56d1774c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.700284 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042a724c-483f-405c-be9e-75a56d1774c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:35:24.700586 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042a724c-483f-405c-be9e-75a56d1774c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:24.702247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042a724c-483f-405c-be9e-75a56d1774c6 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:24.706004 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d7e22259-7b68-4327-a266-afb74f31e864 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:24.707198 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d7e22259-7b68-4327-a266-afb74f31e864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:24.714033 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 624/1243] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:35:24.714806 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a724c-483f-405c-be9e-75a56d1774c6 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:24.718728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 620/1244] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:24.718728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e22259-7b68-4327-a266-afb74f31e864 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:24.728505 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a724c-483f-405c-be9e-75a56d1774c6 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:24.728774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e22259-7b68-4327-a266-afb74f31e864 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:24.733064 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95127b5d-b680-4e17-b07b-c628475a3e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:24.733405 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd311bfc-db3f-4836-bc3c-23f6403ee24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:24.733707 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95127b5d-b680-4e17-b07b-c628475a3e76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.733993 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95127b5d-b680-4e17-b07b-c628475a3e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.734109 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd311bfc-db3f-4836-bc3c-23f6403ee24c admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.734260 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95127b5d-b680-4e17-b07b-c628475a3e76 admin admin] PATH_INFO: `/v3/projects/045a50eaf3e54cf3b8463922b7a3ec3a/users/038007fbad164163bbf856424b59cd63/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.734369 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd311bfc-db3f-4836-bc3c-23f6403ee24c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.734529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd311bfc-db3f-4836-bc3c-23f6403ee24c admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.735178 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd311bfc-db3f-4836-bc3c-23f6403ee24c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:35:24.735480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd311bfc-db3f-4836-bc3c-23f6403ee24c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:24.737386 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:24.738206 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd311bfc-db3f-4836-bc3c-23f6403ee24c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:24.759134 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 621/1245] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:35:24.759363 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95127b5d-b680-4e17-b07b-c628475a3e76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=045a50eaf3e54cf3b8463922b7a3ec3a, user_id=038007fbad164163bbf856424b59cd63, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:24.759457 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd311bfc-db3f-4836-bc3c-23f6403ee24c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:24.759566 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95127b5d-b680-4e17-b07b-c628475a3e76 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:24.763249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd311bfc-db3f-4836-bc3c-23f6403ee24c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:24.768382 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4fa9ea-5f72-49b6-bfd4-af413181e567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:24.769058 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fa9ea-5f72-49b6-bfd4-af413181e567 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.769338 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fa9ea-5f72-49b6-bfd4-af413181e567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.769610 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4fa9ea-5f72-49b6-bfd4-af413181e567 admin admin] PATH_INFO: `/v3/projects/3d4302a426b84e7696c67066b62c1fee/users/47ea84f99dc2419581cf8fa21e26f823/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.771498 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 625/1246] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:24 2022] PUT /identity/v3/projects/045a50eaf3e54cf3b8463922b7a3ec3a/users/038007fbad164163bbf856424b59cd63/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:24.772417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95127b5d-b680-4e17-b07b-c628475a3e76 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:24.785089 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4fa9ea-5f72-49b6-bfd4-af413181e567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d4302a426b84e7696c67066b62c1fee, user_id=47ea84f99dc2419581cf8fa21e26f823, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:24.788678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4fa9ea-5f72-49b6-bfd4-af413181e567 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:24.802003 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 622/1247] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:24 2022] PUT /identity/v3/projects/3d4302a426b84e7696c67066b62c1fee/users/47ea84f99dc2419581cf8fa21e26f823/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:24.803053 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4fa9ea-5f72-49b6-bfd4-af413181e567 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:24.845806 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95127b5d-b680-4e17-b07b-c628475a3e76 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:24.864958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4fa9ea-5f72-49b6-bfd4-af413181e567 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:24.869804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c039c1-0ae6-4c8b-87ca-0380cc336a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:24.870409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c039c1-0ae6-4c8b-87ca-0380cc336a33 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.870685 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c039c1-0ae6-4c8b-87ca-0380cc336a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.870956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c039c1-0ae6-4c8b-87ca-0380cc336a33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.871580 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c039c1-0ae6-4c8b-87ca-0380cc336a33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:24.873586 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c039c1-0ae6-4c8b-87ca-0380cc336a33 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:24.902067 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 623/1248] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:35:24 2022] POST /identity/v3/projects => generated 401 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:24.902968 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c039c1-0ae6-4c8b-87ca-0380cc336a33 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:24.906854 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16497d69-57f2-4c88-9ba2-c91e1cac87f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:24.907580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16497d69-57f2-4c88-9ba2-c91e1cac87f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.907963 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16497d69-57f2-4c88-9ba2-c91e1cac87f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.908339 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16497d69-57f2-4c88-9ba2-c91e1cac87f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.909304 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16497d69-57f2-4c88-9ba2-c91e1cac87f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:35:24.909621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16497d69-57f2-4c88-9ba2-c91e1cac87f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:24.911238 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16497d69-57f2-4c88-9ba2-c91e1cac87f3 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:24.923598 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 626/1249] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:35:24.931287 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16497d69-57f2-4c88-9ba2-c91e1cac87f3 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:24.935952 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16497d69-57f2-4c88-9ba2-c91e1cac87f3 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:24.966387 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c039c1-0ae6-4c8b-87ca-0380cc336a33 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:24.971736 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6fa99-0240-427c-bac9-034138811403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:24.972349 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6fa99-0240-427c-bac9-034138811403 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.972626 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6fa99-0240-427c-bac9-034138811403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.972923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6fa99-0240-427c-bac9-034138811403 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.973968 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6fa99-0240-427c-bac9-034138811403 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:35:24.974272 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6fa99-0240-427c-bac9-034138811403 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:24.976506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6fa99-0240-427c-bac9-034138811403 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:24.991530 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 624/1250] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:35:24.991530 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6fa99-0240-427c-bac9-034138811403 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:24.994179 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75cfe61-552e-490e-ba1d-81f96f689c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:24.994797 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cfe61-552e-490e-ba1d-81f96f689c4d admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:24.995082 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cfe61-552e-490e-ba1d-81f96f689c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:24.995382 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cfe61-552e-490e-ba1d-81f96f689c4d admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:24.996268 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75cfe61-552e-490e-ba1d-81f96f689c4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:35:24.996585 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75cfe61-552e-490e-ba1d-81f96f689c4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:24.998243 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75cfe61-552e-490e-ba1d-81f96f689c4d admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:25.005422 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6fa99-0240-427c-bac9-034138811403 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:25.010396 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4a583-3ac7-4db8-a373-2f608dcb3c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:25.011021 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4a583-3ac7-4db8-a373-2f608dcb3c10 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.011305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4a583-3ac7-4db8-a373-2f608dcb3c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.011936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 627/1251] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:35:25.012699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75cfe61-552e-490e-ba1d-81f96f689c4d admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:25.015207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4a583-3ac7-4db8-a373-2f608dcb3c10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.016262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4a583-3ac7-4db8-a373-2f608dcb3c10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:25.018209 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4a583-3ac7-4db8-a373-2f608dcb3c10 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:25.028469 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75cfe61-552e-490e-ba1d-81f96f689c4d admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:25.033100 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07c128-daf8-4bcf-a868-ba2a78ac7dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:25.033728 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07c128-daf8-4bcf-a868-ba2a78ac7dd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.034008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07c128-daf8-4bcf-a868-ba2a78ac7dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.034458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07c128-daf8-4bcf-a868-ba2a78ac7dd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.035094 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f07c128-daf8-4bcf-a868-ba2a78ac7dd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:25.037106 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f07c128-daf8-4bcf-a868-ba2a78ac7dd5 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:25.051856 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 625/1252] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:35:24 2022] POST /identity/v3/projects => generated 405 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:25.051856 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b488e61-e8ef-4e58-82a0-9872bf9c5869 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.051856 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b488e61-e8ef-4e58-82a0-9872bf9c5869 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.051856 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b488e61-e8ef-4e58-82a0-9872bf9c5869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.057464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:25.062022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:25.073058 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 628/1253] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:35:25 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:25.073058 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d09ee-67d7-4bc6-8c0b-fbdab4c62e46 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.073058 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d09ee-67d7-4bc6-8c0b-fbdab4c62e46 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.073058 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d09ee-67d7-4bc6-8c0b-fbdab4c62e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.080594 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-3b488e61-e8ef-4e58-82a0-9872bf9c5869 None None] MFA Rules not processed for user `038007fbad164163bbf856424b59cd63`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:25.082774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3b488e61-e8ef-4e58-82a0-9872bf9c5869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:25.101456 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-a69d09ee-67d7-4bc6-8c0b-fbdab4c62e46 None None] MFA Rules not processed for user `47ea84f99dc2419581cf8fa21e26f823`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:25.102635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-a69d09ee-67d7-4bc6-8c0b-fbdab4c62e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:25.151402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 626/1254] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:25.152145 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b488e61-e8ef-4e58-82a0-9872bf9c5869 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:25.167811 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 629/1255] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:25.169014 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d09ee-67d7-4bc6-8c0b-fbdab4c62e46 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:25.220582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b488e61-e8ef-4e58-82a0-9872bf9c5869 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:25.225468 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd06af-9492-4a82-b0ca-804ce7d566a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:25.225977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd06af-9492-4a82-b0ca-804ce7d566a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.226260 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd06af-9492-4a82-b0ca-804ce7d566a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.226526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd06af-9492-4a82-b0ca-804ce7d566a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.227152 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd06af-9492-4a82-b0ca-804ce7d566a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:25.229070 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd06af-9492-4a82-b0ca-804ce7d566a6 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:25.240770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d09ee-67d7-4bc6-8c0b-fbdab4c62e46 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:25.246030 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495f173-54e9-48a5-8808-804b92c4c490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:25.246638 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495f173-54e9-48a5-8808-804b92c4c490 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.246921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495f173-54e9-48a5-8808-804b92c4c490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.247194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495f173-54e9-48a5-8808-804b92c4c490 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.247827 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e495f173-54e9-48a5-8808-804b92c4c490 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:25.249792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e495f173-54e9-48a5-8808-804b92c4c490 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:25.253041 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 627/1256] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:35:25 2022] POST /identity/v3/users => generated 301 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:25.253041 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd06af-9492-4a82-b0ca-804ce7d566a6 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:25.263115 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd06af-9492-4a82-b0ca-804ce7d566a6 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:25.268254 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924990d-4d8e-4d3d-80d7-2c50fc2d1ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:25.268919 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924990d-4d8e-4d3d-80d7-2c50fc2d1ca3 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.269236 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924990d-4d8e-4d3d-80d7-2c50fc2d1ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.269532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924990d-4d8e-4d3d-80d7-2c50fc2d1ca3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.270210 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924990d-4d8e-4d3d-80d7-2c50fc2d1ca3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:25.272382 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924990d-4d8e-4d3d-80d7-2c50fc2d1ca3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:25.277989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 630/1257] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:35:25 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:25.281360 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495f173-54e9-48a5-8808-804b92c4c490 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:25.300681 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 628/1258] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:35:25 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:25.305478 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924990d-4d8e-4d3d-80d7-2c50fc2d1ca3 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:25.344246 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495f173-54e9-48a5-8808-804b92c4c490 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:25.361497 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:25.364828 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924990d-4d8e-4d3d-80d7-2c50fc2d1ca3 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:25.369800 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51637b-cbe6-4871-a641-4616f37d9cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:25.370548 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51637b-cbe6-4871-a641-4616f37d9cee admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.370834 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51637b-cbe6-4871-a641-4616f37d9cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.371108 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51637b-cbe6-4871-a641-4616f37d9cee admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.371733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a51637b-cbe6-4871-a641-4616f37d9cee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:25.373680 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a51637b-cbe6-4871-a641-4616f37d9cee admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:25.382626 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:25.399623 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 629/1259] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:35:25 2022] POST /identity/v3/users => generated 303 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:25.400423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51637b-cbe6-4871-a641-4616f37d9cee admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:25.408597 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77c0a8-e3e1-4591-874e-fe5ade0d3e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:25.409183 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77c0a8-e3e1-4591-874e-fe5ade0d3e67 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.409473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77c0a8-e3e1-4591-874e-fe5ade0d3e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.409749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77c0a8-e3e1-4591-874e-fe5ade0d3e67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.409853 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51637b-cbe6-4871-a641-4616f37d9cee admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:25.410436 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77c0a8-e3e1-4591-874e-fe5ade0d3e67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:25.412336 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77c0a8-e3e1-4591-874e-fe5ade0d3e67 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:25.414942 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ce268-8a63-4d82-b606-21e42ca028c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:25.415528 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ce268-8a63-4d82-b606-21e42ca028c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.415811 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ce268-8a63-4d82-b606-21e42ca028c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.416083 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ce268-8a63-4d82-b606-21e42ca028c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.416709 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ce268-8a63-4d82-b606-21e42ca028c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:25.418645 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ce268-8a63-4d82-b606-21e42ca028c3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:25.440022 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 630/1260] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:35:25 2022] POST /identity/v3/users => generated 293 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:25.443903 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 631/1261] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:35:25 2022] POST /identity/v3/projects => generated 377 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:25.444033 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ce268-8a63-4d82-b606-21e42ca028c3 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:25.444114 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07d845-2ca7-4d02-a19b-1da51944ff08 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.444114 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07d845-2ca7-4d02-a19b-1da51944ff08 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.444114 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07d845-2ca7-4d02-a19b-1da51944ff08 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.445474 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 632/1262] 10.222.0.23 () {54 vars in 961 bytes} [Thu Oct 20 13:35:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:35:25.445973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07d845-2ca7-4d02-a19b-1da51944ff08 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:25.499261 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ce268-8a63-4d82-b606-21e42ca028c3 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:25.517126 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07d845-2ca7-4d02-a19b-1da51944ff08 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:25.522227 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f01c3d-6b2c-4b6c-bc6e-33f163f45aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:25.522838 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f01c3d-6b2c-4b6c-bc6e-33f163f45aba admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.523121 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f01c3d-6b2c-4b6c-bc6e-33f163f45aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.523397 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f01c3d-6b2c-4b6c-bc6e-33f163f45aba admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.524077 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f01c3d-6b2c-4b6c-bc6e-33f163f45aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:25.525751 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f01c3d-6b2c-4b6c-bc6e-33f163f45aba admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:25.536262 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 633/1263] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:25.537604 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f01c3d-6b2c-4b6c-bc6e-33f163f45aba admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:25.547355 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f01c3d-6b2c-4b6c-bc6e-33f163f45aba admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:25.550115 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf4b1f-8cce-4105-878a-f22f0ae2afd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:25.550722 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf4b1f-8cce-4105-878a-f22f0ae2afd0 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.551000 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf4b1f-8cce-4105-878a-f22f0ae2afd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.551282 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf4b1f-8cce-4105-878a-f22f0ae2afd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.552729 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17d5ae-17a2-4114-8c11-b0f72a5e05b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:25.553877 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17d5ae-17a2-4114-8c11-b0f72a5e05b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.554157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17d5ae-17a2-4114-8c11-b0f72a5e05b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.554432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17d5ae-17a2-4114-8c11-b0f72a5e05b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.555058 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a17d5ae-17a2-4114-8c11-b0f72a5e05b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:25.557051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a17d5ae-17a2-4114-8c11-b0f72a5e05b8 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:25.578525 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 634/1264] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:35:25 2022] POST /identity/v3/users => generated 294 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:25.579327 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17d5ae-17a2-4114-8c11-b0f72a5e05b8 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:25.590010 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17d5ae-17a2-4114-8c11-b0f72a5e05b8 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:25.595159 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587e9e4-06cd-4a66-b21f-baba60483fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:25.595749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587e9e4-06cd-4a66-b21f-baba60483fe8 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.596030 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587e9e4-06cd-4a66-b21f-baba60483fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.596309 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587e9e4-06cd-4a66-b21f-baba60483fe8 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.597904 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587e9e4-06cd-4a66-b21f-baba60483fe8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:25.600143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587e9e4-06cd-4a66-b21f-baba60483fe8 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:25.609336 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cf4b1f-8cce-4105-878a-f22f0ae2afd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:25.610757 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cf4b1f-8cce-4105-878a-f22f0ae2afd0 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:25.621003 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 631/1265] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:25.622121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf4b1f-8cce-4105-878a-f22f0ae2afd0 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:25.626287 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 635/1266] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:35:25 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:25.626287 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587e9e4-06cd-4a66-b21f-baba60483fe8 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:25.635386 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587e9e4-06cd-4a66-b21f-baba60483fe8 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:25.637883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf4b1f-8cce-4105-878a-f22f0ae2afd0 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:25.640153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f656718-4675-49e2-9255-0791c42d8d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:25.640755 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f656718-4675-49e2-9255-0791c42d8d9c admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.641069 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f656718-4675-49e2-9255-0791c42d8d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.641342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f656718-4675-49e2-9255-0791c42d8d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.641986 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f656718-4675-49e2-9255-0791c42d8d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:25.643552 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ad3d9-99dc-4f2f-8cac-3f1a6404a9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:25.643689 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f656718-4675-49e2-9255-0791c42d8d9c admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:25.644270 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ad3d9-99dc-4f2f-8cac-3f1a6404a9aa admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.644623 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ad3d9-99dc-4f2f-8cac-3f1a6404a9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.645104 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ad3d9-99dc-4f2f-8cac-3f1a6404a9aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.645808 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ad3d9-99dc-4f2f-8cac-3f1a6404a9aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:25.647608 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ad3d9-99dc-4f2f-8cac-3f1a6404a9aa admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:25.655448 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 636/1267] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:25.656413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f656718-4675-49e2-9255-0791c42d8d9c admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:25.662236 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 632/1268] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:25.663452 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3ebc0-21bd-4ca4-9b18-b8fe2bb55c3e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.663739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3ebc0-21bd-4ca4-9b18-b8fe2bb55c3e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.664011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3ebc0-21bd-4ca4-9b18-b8fe2bb55c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.670048 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f656718-4675-49e2-9255-0791c42d8d9c admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:25.676010 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097fcab-5b76-49eb-bd12-ac8ec72c07e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:25.676609 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097fcab-5b76-49eb-bd12-ac8ec72c07e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.676954 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097fcab-5b76-49eb-bd12-ac8ec72c07e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.678252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097fcab-5b76-49eb-bd12-ac8ec72c07e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.678909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5097fcab-5b76-49eb-bd12-ac8ec72c07e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:25.681868 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5097fcab-5b76-49eb-bd12-ac8ec72c07e7 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:25.704806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c7d3ebc0-21bd-4ca4-9b18-b8fe2bb55c3e None None] MFA Rules not processed for user `ee5c2a3855cb4a668177aa19f337778d`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:25.706240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c7d3ebc0-21bd-4ca4-9b18-b8fe2bb55c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:25.708435 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 637/1269] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:35:25 2022] POST /identity/v3/users => generated 289 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:25.712680 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097fcab-5b76-49eb-bd12-ac8ec72c07e7 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:25.721477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:25.724102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097fcab-5b76-49eb-bd12-ac8ec72c07e7 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:25.730626 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0944956-4a60-493b-ac97-e7dd4d4528e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:25.731233 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0944956-4a60-493b-ac97-e7dd4d4528e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.731648 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0944956-4a60-493b-ac97-e7dd4d4528e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.731932 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0944956-4a60-493b-ac97-e7dd4d4528e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.732705 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0944956-4a60-493b-ac97-e7dd4d4528e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:25.734334 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0944956-4a60-493b-ac97-e7dd4d4528e0 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:25.746884 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 638/1270] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:25.747532 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0944956-4a60-493b-ac97-e7dd4d4528e0 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:25.758662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0944956-4a60-493b-ac97-e7dd4d4528e0 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:25.763442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a870ac7-fa4f-47ae-b107-6c516ea6c6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:25.764137 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a870ac7-fa4f-47ae-b107-6c516ea6c6d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.764500 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a870ac7-fa4f-47ae-b107-6c516ea6c6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.764835 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a870ac7-fa4f-47ae-b107-6c516ea6c6d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.765566 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a870ac7-fa4f-47ae-b107-6c516ea6c6d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:25.767220 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a870ac7-fa4f-47ae-b107-6c516ea6c6d6 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:25.780892 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 639/1271] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:25.780892 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a870ac7-fa4f-47ae-b107-6c516ea6c6d6 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:25.789058 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a870ac7-fa4f-47ae-b107-6c516ea6c6d6 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:25.793491 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 633/1272] 10.222.0.23 () {58 vars in 1058 bytes} [Thu Oct 20 13:35:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:25.794391 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3ebc0-21bd-4ca4-9b18-b8fe2bb55c3e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:25.800567 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16f5da-809b-40e1-b6a2-a5344c249a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:25.801518 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16f5da-809b-40e1-b6a2-a5344c249a1d admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.801806 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16f5da-809b-40e1-b6a2-a5344c249a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.802082 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16f5da-809b-40e1-b6a2-a5344c249a1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.803296 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e16f5da-809b-40e1-b6a2-a5344c249a1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:25.804962 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e16f5da-809b-40e1-b6a2-a5344c249a1d admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:25.806398 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3ebc0-21bd-4ca4-9b18-b8fe2bb55c3e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:25.811154 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bac97c-8f80-4f5d-9f38-628e0dae665b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:25.811748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bac97c-8f80-4f5d-9f38-628e0dae665b admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.812023 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bac97c-8f80-4f5d-9f38-628e0dae665b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.812305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bac97c-8f80-4f5d-9f38-628e0dae665b admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.812954 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bac97c-8f80-4f5d-9f38-628e0dae665b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:25.814524 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bac97c-8f80-4f5d-9f38-628e0dae665b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:25.815766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 640/1273] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:25.816510 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16f5da-809b-40e1-b6a2-a5344c249a1d admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:25.823165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:25.828909 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 634/1274] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:25.828909 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bac97c-8f80-4f5d-9f38-628e0dae665b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:25.829659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16f5da-809b-40e1-b6a2-a5344c249a1d admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:25.839924 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d083251-b616-4634-9909-a52b6aefa145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:25.840552 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d083251-b616-4634-9909-a52b6aefa145 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.840964 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d083251-b616-4634-9909-a52b6aefa145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.841135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d083251-b616-4634-9909-a52b6aefa145 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.841279 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bac97c-8f80-4f5d-9f38-628e0dae665b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:25.841778 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d083251-b616-4634-9909-a52b6aefa145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:25.843362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d083251-b616-4634-9909-a52b6aefa145 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:25.846046 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7be3b-1247-4d31-b82b-57353666ffb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:25.846651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7be3b-1247-4d31-b82b-57353666ffb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.846932 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7be3b-1247-4d31-b82b-57353666ffb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.847205 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7be3b-1247-4d31-b82b-57353666ffb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.847861 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7be3b-1247-4d31-b82b-57353666ffb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:25.849504 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7be3b-1247-4d31-b82b-57353666ffb3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:25.857034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 641/1275] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:25.857885 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d083251-b616-4634-9909-a52b6aefa145 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:25.860451 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 635/1276] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:25.861309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7be3b-1247-4d31-b82b-57353666ffb3 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:25.871160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d083251-b616-4634-9909-a52b6aefa145 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:25.873768 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7be3b-1247-4d31-b82b-57353666ffb3 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:25.877246 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d7806-1f62-4ef0-9e10-cb9907a33f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:25.878572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f31a9b-cd6a-4f3f-810d-1094297fb4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:25.878700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d7806-1f62-4ef0-9e10-cb9907a33f65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.879188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d7806-1f62-4ef0-9e10-cb9907a33f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.879440 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f31a9b-cd6a-4f3f-810d-1094297fb4ba admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.879542 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d7806-1f62-4ef0-9e10-cb9907a33f65 admin admin] PATH_INFO: `/v3/projects/861546743d064fcf94b72c6eb5e3b233/users/a49d0c317d3d4e4aa7b98e5167061c11/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.879693 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f31a9b-cd6a-4f3f-810d-1094297fb4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.879949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f31a9b-cd6a-4f3f-810d-1094297fb4ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.880609 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f31a9b-cd6a-4f3f-810d-1094297fb4ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:25.882363 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f31a9b-cd6a-4f3f-810d-1094297fb4ba admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:25.895024 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 636/1277] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:25.896038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f31a9b-cd6a-4f3f-810d-1094297fb4ba admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:25.900135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d7806-1f62-4ef0-9e10-cb9907a33f65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=861546743d064fcf94b72c6eb5e3b233, user_id=a49d0c317d3d4e4aa7b98e5167061c11, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:25.903033 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d7806-1f62-4ef0-9e10-cb9907a33f65 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:25.910160 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f31a9b-cd6a-4f3f-810d-1094297fb4ba admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:25.918635 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd92bd-e355-4255-b2ec-a90e5ef0b815 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:25.918871 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 642/1278] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:25 2022] PUT /identity/v3/projects/861546743d064fcf94b72c6eb5e3b233/users/a49d0c317d3d4e4aa7b98e5167061c11/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:25.919537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd92bd-e355-4255-b2ec-a90e5ef0b815 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:25.919819 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd92bd-e355-4255-b2ec-a90e5ef0b815 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:25.919947 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d7806-1f62-4ef0-9e10-cb9907a33f65 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:25.920089 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd92bd-e355-4255-b2ec-a90e5ef0b815 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:25.989422 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd92bd-e355-4255-b2ec-a90e5ef0b815 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:25.991272 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd92bd-e355-4255-b2ec-a90e5ef0b815 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:26.000536 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d7806-1f62-4ef0-9e10-cb9907a33f65 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:26.001242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 637/1279] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:26.006285 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aeaa04-1238-4ef0-b9a7-cd128e07524c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:26.007130 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aeaa04-1238-4ef0-b9a7-cd128e07524c admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:26.007489 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aeaa04-1238-4ef0-b9a7-cd128e07524c admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:26.007837 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aeaa04-1238-4ef0-b9a7-cd128e07524c admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:26.008591 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aeaa04-1238-4ef0-b9a7-cd128e07524c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:26.009903 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd92bd-e355-4255-b2ec-a90e5ef0b815 service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:26.010415 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aeaa04-1238-4ef0-b9a7-cd128e07524c admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:26.021415 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 643/1280] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:26.023089 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd92bd-e355-4255-b2ec-a90e5ef0b815 service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:26.025895 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aeaa04-1238-4ef0-b9a7-cd128e07524c admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:26.029458 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56d9b53-4b17-4719-9ee3-434334c03f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:26.030426 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d9b53-4b17-4719-9ee3-434334c03f42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:26.030860 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d9b53-4b17-4719-9ee3-434334c03f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:26.031594 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d9b53-4b17-4719-9ee3-434334c03f42 admin admin] PATH_INFO: `/v3/projects/b215cb88123a4851932dbf7dfa88f709/users/144e9801ac1f470083d11647652ccdf1/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:26.036883 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aeaa04-1238-4ef0-b9a7-cd128e07524c admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:26.042027 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db08140-7d95-4e76-b2b0-34f644c2f152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:26.042770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db08140-7d95-4e76-b2b0-34f644c2f152 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:26.043176 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db08140-7d95-4e76-b2b0-34f644c2f152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:26.043528 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db08140-7d95-4e76-b2b0-34f644c2f152 admin admin] PATH_INFO: `/v3/projects/4f8ffcea74224edc99bd7c9fd4b796ec/users/dbfb17284ffe448cb2e2c0713123464b/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:26.053964 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56d9b53-4b17-4719-9ee3-434334c03f42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b215cb88123a4851932dbf7dfa88f709, user_id=144e9801ac1f470083d11647652ccdf1, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:26.059005 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56d9b53-4b17-4719-9ee3-434334c03f42 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:26.061010 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db08140-7d95-4e76-b2b0-34f644c2f152 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f8ffcea74224edc99bd7c9fd4b796ec, user_id=dbfb17284ffe448cb2e2c0713123464b, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:26.064102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db08140-7d95-4e76-b2b0-34f644c2f152 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:26.082034 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 638/1282] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:26 2022] PUT /identity/v3/projects/b215cb88123a4851932dbf7dfa88f709/users/144e9801ac1f470083d11647652ccdf1/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:26.082275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 644/1282] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:26 2022] PUT /identity/v3/projects/4f8ffcea74224edc99bd7c9fd4b796ec/users/dbfb17284ffe448cb2e2c0713123464b/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:26.082379 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56d9b53-4b17-4719-9ee3-434334c03f42 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:26.082471 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db08140-7d95-4e76-b2b0-34f644c2f152 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:26.176254 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:26.196342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db08140-7d95-4e76-b2b0-34f644c2f152 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:26.200029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56d9b53-4b17-4719-9ee3-434334c03f42 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:26.202668 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a360c5-569b-4d6a-a13c-ca7b97da03b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:26.203373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a360c5-569b-4d6a-a13c-ca7b97da03b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:26.203750 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a360c5-569b-4d6a-a13c-ca7b97da03b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:26.204121 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a360c5-569b-4d6a-a13c-ca7b97da03b3 admin admin] PATH_INFO: `/v3/projects/f9addc326c51410f809805d8b44c5c17/users/55526ddbea014e6cbf7c079cb204059e/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:26.208527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16edaa2-773a-4dbf-9f7b-705208db1adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:26.209344 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16edaa2-773a-4dbf-9f7b-705208db1adb admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:26.209700 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16edaa2-773a-4dbf-9f7b-705208db1adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:26.210047 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16edaa2-773a-4dbf-9f7b-705208db1adb admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:26.210780 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16edaa2-773a-4dbf-9f7b-705208db1adb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:26.212456 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16edaa2-773a-4dbf-9f7b-705208db1adb admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:26.229316 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a360c5-569b-4d6a-a13c-ca7b97da03b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9addc326c51410f809805d8b44c5c17, user_id=55526ddbea014e6cbf7c079cb204059e, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:26.231984 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a360c5-569b-4d6a-a13c-ca7b97da03b3 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:26.241969 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 639/1283] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:26.243496 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c920e-99d5-477a-81ff-4e8b020824b2 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:26.243735 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c920e-99d5-477a-81ff-4e8b020824b2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:26.244165 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c920e-99d5-477a-81ff-4e8b020824b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:26.254925 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 645/1284] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:26 2022] PUT /identity/v3/projects/f9addc326c51410f809805d8b44c5c17/users/55526ddbea014e6cbf7c079cb204059e/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:26.256954 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a360c5-569b-4d6a-a13c-ca7b97da03b3 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:26.287852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b35c920e-99d5-477a-81ff-4e8b020824b2 None None] MFA Rules not processed for user `a49d0c317d3d4e4aa7b98e5167061c11`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:26.289344 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b35c920e-99d5-477a-81ff-4e8b020824b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:26.305093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:26.349793 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a360c5-569b-4d6a-a13c-ca7b97da03b3 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:26.355097 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65af61-d502-4bf2-be79-3e62075597d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:26.355771 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65af61-d502-4bf2-be79-3e62075597d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:26.356211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65af61-d502-4bf2-be79-3e62075597d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:26.356557 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65af61-d502-4bf2-be79-3e62075597d4 admin admin] PATH_INFO: `/v3/projects/4e1ecd40d6914036b80dfcf74eada323/users/6085945ca3304bc99b4d9ca8a37d5319/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:26.364671 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 640/1285] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:26 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:26.367182 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767a6b0-fb9a-48bc-804f-2e43618b86bd None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:26.367691 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767a6b0-fb9a-48bc-804f-2e43618b86bd None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:26.368067 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767a6b0-fb9a-48bc-804f-2e43618b86bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:26.373950 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff65af61-d502-4bf2-be79-3e62075597d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e1ecd40d6914036b80dfcf74eada323, user_id=6085945ca3304bc99b4d9ca8a37d5319, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:26.377399 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff65af61-d502-4bf2-be79-3e62075597d4 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:26.392815 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8767a6b0-fb9a-48bc-804f-2e43618b86bd None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:26.394181 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8767a6b0-fb9a-48bc-804f-2e43618b86bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:26.401549 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 646/1286] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:26 2022] PUT /identity/v3/projects/4e1ecd40d6914036b80dfcf74eada323/users/6085945ca3304bc99b4d9ca8a37d5319/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:26.403030 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00bfd3-ca30-4942-88a1-a6dc7f88d95a None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:26.403323 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00bfd3-ca30-4942-88a1-a6dc7f88d95a None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:26.403601 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00bfd3-ca30-4942-88a1-a6dc7f88d95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:26.435452 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-1e00bfd3-ca30-4942-88a1-a6dc7f88d95a None None] MFA Rules not processed for user `dbfb17284ffe448cb2e2c0713123464b`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:26.436639 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-1e00bfd3-ca30-4942-88a1-a6dc7f88d95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:26.468050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 641/1287] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:26.468050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767a6b0-fb9a-48bc-804f-2e43618b86bd None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:26.476332 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767a6b0-fb9a-48bc-804f-2e43618b86bd None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:26.481284 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26f514-9168-40fb-989d-60444bcd7c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:26.482009 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26f514-9168-40fb-989d-60444bcd7c34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:26.482281 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26f514-9168-40fb-989d-60444bcd7c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:26.482587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26f514-9168-40fb-989d-60444bcd7c34 admin admin] PATH_INFO: `/v3/projects/747a468811fb478aa6c49b848ccee6d0/users/fe56d1d4278c43cd96a39f7f75f373b8/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:26.501901 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26f514-9168-40fb-989d-60444bcd7c34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=747a468811fb478aa6c49b848ccee6d0, user_id=fe56d1d4278c43cd96a39f7f75f373b8, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:26.504905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26f514-9168-40fb-989d-60444bcd7c34 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:26.518520 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 642/1288] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:26 2022] PUT /identity/v3/projects/747a468811fb478aa6c49b848ccee6d0/users/fe56d1d4278c43cd96a39f7f75f373b8/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:26.522355 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 647/1289] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:26 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:26.523560 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00bfd3-ca30-4942-88a1-a6dc7f88d95a None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:26.526505 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924281b4-2215-4874-a70a-ca0eee310ffb None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:26.527367 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924281b4-2215-4874-a70a-ca0eee310ffb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:26.527651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924281b4-2215-4874-a70a-ca0eee310ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:26.537399 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00bfd3-ca30-4942-88a1-a6dc7f88d95a None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:26.550248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-924281b4-2215-4874-a70a-ca0eee310ffb None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:26.550248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-924281b4-2215-4874-a70a-ca0eee310ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:26.607002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a703be-e49f-4d60-8887-167763f3fb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:26.607937 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a703be-e49f-4d60-8887-167763f3fb58 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:26.608223 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a703be-e49f-4d60-8887-167763f3fb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:26.608495 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a703be-e49f-4d60-8887-167763f3fb58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:26.609212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a703be-e49f-4d60-8887-167763f3fb58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:26.611092 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a703be-e49f-4d60-8887-167763f3fb58 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:26.614807 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:26.634932 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 643/1290] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:26.636272 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598d8d1-38e3-4877-8ee2-9d8fb0b21c8f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:26.636557 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598d8d1-38e3-4877-8ee2-9d8fb0b21c8f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:26.636845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598d8d1-38e3-4877-8ee2-9d8fb0b21c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:26.645058 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 648/1291] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:35:26 2022] POST /identity/v3/projects => generated 401 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:26.645058 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a703be-e49f-4d60-8887-167763f3fb58 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:26.657666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1598d8d1-38e3-4877-8ee2-9d8fb0b21c8f None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:26.658903 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1598d8d1-38e3-4877-8ee2-9d8fb0b21c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:26.671413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:26.702706 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a703be-e49f-4d60-8887-167763f3fb58 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:26.706256 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ddc43-7e52-4f40-a260-72038b94a89e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:26.706897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ddc43-7e52-4f40-a260-72038b94a89e service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:26.707143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ddc43-7e52-4f40-a260-72038b94a89e service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:26.707414 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ddc43-7e52-4f40-a260-72038b94a89e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:26.718131 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 644/1292] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:26.719450 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7f1e0-5f46-4c18-9c2a-d9da8466b48c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:26.719726 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7f1e0-5f46-4c18-9c2a-d9da8466b48c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:26.720008 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7f1e0-5f46-4c18-9c2a-d9da8466b48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:26.738602 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7ed7f1e0-5f46-4c18-9c2a-d9da8466b48c None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:26.739784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7ed7f1e0-5f46-4c18-9c2a-d9da8466b48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:26.747854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 645/1293] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:26.751719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7f1e0-5f46-4c18-9c2a-d9da8466b48c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:26.762346 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7f1e0-5f46-4c18-9c2a-d9da8466b48c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:26.766973 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56ab70-85b8-4fa2-8976-5bc1cb5bcbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:26.767566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56ab70-85b8-4fa2-8976-5bc1cb5bcbce admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:26.767840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56ab70-85b8-4fa2-8976-5bc1cb5bcbce admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:26.768109 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56ab70-85b8-4fa2-8976-5bc1cb5bcbce admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:26.768732 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56ab70-85b8-4fa2-8976-5bc1cb5bcbce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:26.770665 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56ab70-85b8-4fa2-8976-5bc1cb5bcbce admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:26.773467 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ddc43-7e52-4f40-a260-72038b94a89e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:26.774909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ddc43-7e52-4f40-a260-72038b94a89e service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:26.784567 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 649/1294] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:26.786020 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4e076-5811-41a4-9886-ef4958a32494 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:26.786313 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4e076-5811-41a4-9886-ef4958a32494 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:26.786592 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4e076-5811-41a4-9886-ef4958a32494 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:26.788418 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 650/1295] 10.222.0.23 () {54 vars in 961 bytes} [Thu Oct 20 13:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:35:26.789226 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4e076-5811-41a4-9886-ef4958a32494 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:26.800287 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4e076-5811-41a4-9886-ef4958a32494 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:26.805100 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea6632-efeb-4b6c-bdee-275b7919ce12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:26.805692 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea6632-efeb-4b6c-bdee-275b7919ce12 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:26.806003 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea6632-efeb-4b6c-bdee-275b7919ce12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:26.806478 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea6632-efeb-4b6c-bdee-275b7919ce12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:26.810543 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 646/1296] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:35:26 2022] POST /identity/v3/projects => generated 403 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:26.810543 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56ab70-85b8-4fa2-8976-5bc1cb5bcbce admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:26.819775 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea6632-efeb-4b6c-bdee-275b7919ce12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:26.821269 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea6632-efeb-4b6c-bdee-275b7919ce12 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:26.831400 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 651/1297] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:26.832283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea6632-efeb-4b6c-bdee-275b7919ce12 service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:26.833564 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-f0ea6632-efeb-4b6c-bdee-275b7919ce12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:26.845475 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea6632-efeb-4b6c-bdee-275b7919ce12 service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:26.885119 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56ab70-85b8-4fa2-8976-5bc1cb5bcbce admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:26.890230 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e93f8-45a7-4f4d-ace4-cdfe618c9407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:26.890989 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e93f8-45a7-4f4d-ace4-cdfe618c9407 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:26.891290 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e93f8-45a7-4f4d-ace4-cdfe618c9407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:26.891588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e93f8-45a7-4f4d-ace4-cdfe618c9407 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:26.892224 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e93f8-45a7-4f4d-ace4-cdfe618c9407 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:26.894246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e93f8-45a7-4f4d-ace4-cdfe618c9407 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:26.906557 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433d9fe-6425-437d-ac68-220b5c850476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:26.908432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433d9fe-6425-437d-ac68-220b5c850476 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:26.908722 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433d9fe-6425-437d-ac68-220b5c850476 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:26.909234 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433d9fe-6425-437d-ac68-220b5c850476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:26.920589 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 647/1298] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:35:26 2022] POST /identity/v3/users => generated 307 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:26.921568 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e93f8-45a7-4f4d-ace4-cdfe618c9407 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:26.926719 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433d9fe-6425-437d-ac68-220b5c850476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:26.929602 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433d9fe-6425-437d-ac68-220b5c850476 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:26.932613 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e93f8-45a7-4f4d-ace4-cdfe618c9407 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:26.936405 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfd87b-f784-4dac-a2ba-eb21ddced598 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:26.937059 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfd87b-f784-4dac-a2ba-eb21ddced598 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:26.937413 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfd87b-f784-4dac-a2ba-eb21ddced598 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:26.937697 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfd87b-f784-4dac-a2ba-eb21ddced598 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:26.949142 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 652/1299] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:26.950592 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecfd87b-f784-4dac-a2ba-eb21ddced598 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:26.952122 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecfd87b-f784-4dac-a2ba-eb21ddced598 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:26.952698 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291013a-2734-4369-8943-e36629d81c00 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:26.953334 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291013a-2734-4369-8943-e36629d81c00 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:26.953849 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291013a-2734-4369-8943-e36629d81c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:26.963447 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 648/1300] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:26.964365 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfd87b-f784-4dac-a2ba-eb21ddced598 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:26.977050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfd87b-f784-4dac-a2ba-eb21ddced598 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:26.982274 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67a7ea-9739-4232-a143-cf4d0fc04473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:26.982782 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-c291013a-2734-4369-8943-e36629d81c00 None None] MFA Rules not processed for user `ee5c2a3855cb4a668177aa19f337778d`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:26.982950 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67a7ea-9739-4232-a143-cf4d0fc04473 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:26.983236 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67a7ea-9739-4232-a143-cf4d0fc04473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:26.983510 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67a7ea-9739-4232-a143-cf4d0fc04473 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:26.984224 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb67a7ea-9739-4232-a143-cf4d0fc04473 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:26.984371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-c291013a-2734-4369-8943-e36629d81c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:26.986311 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb67a7ea-9739-4232-a143-cf4d0fc04473 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:27.016886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 649/1301] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:35:26 2022] POST /identity/v3/users => generated 308 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:27.018922 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67a7ea-9739-4232-a143-cf4d0fc04473 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:27.034756 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67a7ea-9739-4232-a143-cf4d0fc04473 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:27.044311 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79834ead-7f9a-4356-93a4-b49c6dd20e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:27.045248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79834ead-7f9a-4356-93a4-b49c6dd20e53 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:27.045748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79834ead-7f9a-4356-93a4-b49c6dd20e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:27.046220 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79834ead-7f9a-4356-93a4-b49c6dd20e53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:27.046864 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79834ead-7f9a-4356-93a4-b49c6dd20e53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:27.048982 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79834ead-7f9a-4356-93a4-b49c6dd20e53 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:27.065540 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 650/1302] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:27.069534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79834ead-7f9a-4356-93a4-b49c6dd20e53 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:27.077946 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:27.086107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79834ead-7f9a-4356-93a4-b49c6dd20e53 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:27.091801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55410b-1128-4557-910b-f7fb52902dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:27.092672 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55410b-1128-4557-910b-f7fb52902dfd admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:27.093208 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55410b-1128-4557-910b-f7fb52902dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:27.093657 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55410b-1128-4557-910b-f7fb52902dfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:27.094453 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55410b-1128-4557-910b-f7fb52902dfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:27.096430 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55410b-1128-4557-910b-f7fb52902dfd admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:27.098669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 653/1303] 10.222.0.23 () {58 vars in 1058 bytes} [Thu Oct 20 13:35:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:27.100520 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291013a-2734-4369-8943-e36629d81c00 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:27.111132 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 651/1304] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:27.112937 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59334ac9-95e5-41da-aa4e-c05abe793bb4 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:27.113473 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59334ac9-95e5-41da-aa4e-c05abe793bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:27.113928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59334ac9-95e5-41da-aa4e-c05abe793bb4 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:27.115781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291013a-2734-4369-8943-e36629d81c00 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:27.116169 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 652/1305] 10.222.0.23 () {54 vars in 912 bytes} [Thu Oct 20 13:35:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:35:27.117344 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59334ac9-95e5-41da-aa4e-c05abe793bb4 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:27.122593 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46cb8a6-2bfe-4933-9be5-2e599fa762a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:27.123787 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46cb8a6-2bfe-4933-9be5-2e599fa762a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:27.124175 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46cb8a6-2bfe-4933-9be5-2e599fa762a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:27.124545 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46cb8a6-2bfe-4933-9be5-2e599fa762a5 admin admin] PATH_INFO: `/v3/projects/1216452f661044568b65a1f5370a7a95/users/8bc7c49859bb489ab4d7d94467207ddd/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:27.126955 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:27.142198 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59334ac9-95e5-41da-aa4e-c05abe793bb4 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:27.148181 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46cb8a6-2bfe-4933-9be5-2e599fa762a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1216452f661044568b65a1f5370a7a95, user_id=8bc7c49859bb489ab4d7d94467207ddd, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:27.151681 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46cb8a6-2bfe-4933-9be5-2e599fa762a5 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:27.155179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f62368-77d1-44a9-a71c-379f3d276b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:27.155956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f62368-77d1-44a9-a71c-379f3d276b3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:27.156408 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f62368-77d1-44a9-a71c-379f3d276b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:27.156785 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f62368-77d1-44a9-a71c-379f3d276b3e admin admin] PATH_INFO: `/v3/projects/e29cab0a4c194b63a9ae234afd9256cd/users/ea511efaebcb4b03aaf726688e230291/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:27.183385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 654/1306] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:27 2022] PUT /identity/v3/projects/1216452f661044568b65a1f5370a7a95/users/8bc7c49859bb489ab4d7d94467207ddd/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:27.185168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235cc15-4cf2-4dcd-87b9-770ce0aa03b4 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:27.185376 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f62368-77d1-44a9-a71c-379f3d276b3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e29cab0a4c194b63a9ae234afd9256cd, user_id=ea511efaebcb4b03aaf726688e230291, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:27.186319 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235cc15-4cf2-4dcd-87b9-770ce0aa03b4 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:27.186810 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235cc15-4cf2-4dcd-87b9-770ce0aa03b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:27.188292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f62368-77d1-44a9-a71c-379f3d276b3e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:27.213320 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 653/1307] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:27 2022] PUT /identity/v3/projects/e29cab0a4c194b63a9ae234afd9256cd/users/ea511efaebcb4b03aaf726688e230291/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:27.214637 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f62368-77d1-44a9-a71c-379f3d276b3e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:27.223380 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-1235cc15-4cf2-4dcd-87b9-770ce0aa03b4 None None] MFA Rules not processed for user `ee5c2a3855cb4a668177aa19f337778d`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:27.225460 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-1235cc15-4cf2-4dcd-87b9-770ce0aa03b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:27.319490 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f62368-77d1-44a9-a71c-379f3d276b3e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:27.326386 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022befa-2072-4580-9a62-0f59a7f024c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:27.326866 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022befa-2072-4580-9a62-0f59a7f024c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:27.328237 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022befa-2072-4580-9a62-0f59a7f024c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:27.328237 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022befa-2072-4580-9a62-0f59a7f024c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:27.328467 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022befa-2072-4580-9a62-0f59a7f024c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:27.331085 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022befa-2072-4580-9a62-0f59a7f024c1 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:27.334977 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 655/1308] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 20 13:35:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:27.336681 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235cc15-4cf2-4dcd-87b9-770ce0aa03b4 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:27.350801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 654/1309] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:27.350801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022befa-2072-4580-9a62-0f59a7f024c1 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:27.362214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235cc15-4cf2-4dcd-87b9-770ce0aa03b4 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:27.371630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c64a47-2b57-4235-91d9-7b7275ba8fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:27.372724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c64a47-2b57-4235-91d9-7b7275ba8fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:27.373369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c64a47-2b57-4235-91d9-7b7275ba8fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:27.374112 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c64a47-2b57-4235-91d9-7b7275ba8fd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:27.375097 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c64a47-2b57-4235-91d9-7b7275ba8fd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:27.377067 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c64a47-2b57-4235-91d9-7b7275ba8fd7 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:27.394708 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 656/1310] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:27.397103 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c64a47-2b57-4235-91d9-7b7275ba8fd7 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:27.413275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c64a47-2b57-4235-91d9-7b7275ba8fd7 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:27.422934 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc2389-0ab7-43b2-8c87-a67180a81a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:27.425192 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc2389-0ab7-43b2-8c87-a67180a81a70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:27.425718 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc2389-0ab7-43b2-8c87-a67180a81a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:27.426199 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc2389-0ab7-43b2-8c87-a67180a81a70 admin admin] PATH_INFO: `/v3/projects/1216452f661044568b65a1f5370a7a95/users/8bc7c49859bb489ab4d7d94467207ddd/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:27.428792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc2389-0ab7-43b2-8c87-a67180a81a70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1216452f661044568b65a1f5370a7a95, user_id=8bc7c49859bb489ab4d7d94467207ddd, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:27.431853 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc2389-0ab7-43b2-8c87-a67180a81a70 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:27.463673 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022befa-2072-4580-9a62-0f59a7f024c1 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:27.477347 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 657/1311] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:27 2022] PUT /identity/v3/projects/1216452f661044568b65a1f5370a7a95/users/8bc7c49859bb489ab4d7d94467207ddd/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:27.477347 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc2389-0ab7-43b2-8c87-a67180a81a70 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:27.535130 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3284f-86c6-44c2-a9e5-917d00250451 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:27.535820 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3284f-86c6-44c2-a9e5-917d00250451 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:27.536173 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3284f-86c6-44c2-a9e5-917d00250451 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:27.536552 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3284f-86c6-44c2-a9e5-917d00250451 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:27.582143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc2389-0ab7-43b2-8c87-a67180a81a70 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:27.588047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054a4bd-746d-4e60-9b2a-b413e42cfc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:27.589867 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054a4bd-746d-4e60-9b2a-b413e42cfc48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:27.590160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054a4bd-746d-4e60-9b2a-b413e42cfc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:27.591772 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054a4bd-746d-4e60-9b2a-b413e42cfc48 admin admin] PATH_INFO: `/v3/projects/e29cab0a4c194b63a9ae234afd9256cd/users/ea511efaebcb4b03aaf726688e230291/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:27.593889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3054a4bd-746d-4e60-9b2a-b413e42cfc48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e29cab0a4c194b63a9ae234afd9256cd, user_id=ea511efaebcb4b03aaf726688e230291, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:27.604793 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3054a4bd-746d-4e60-9b2a-b413e42cfc48 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:27.624854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:27.635458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 658/1312] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:27 2022] PUT /identity/v3/projects/e29cab0a4c194b63a9ae234afd9256cd/users/ea511efaebcb4b03aaf726688e230291/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:27.637250 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b7925-ecc4-4b0a-824d-be00966f3475 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:27.637548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b7925-ecc4-4b0a-824d-be00966f3475 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:27.637959 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b7925-ecc4-4b0a-824d-be00966f3475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:27.692825 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-5c7b7925-ecc4-4b0a-824d-be00966f3475 None None] MFA Rules not processed for user `8bc7c49859bb489ab4d7d94467207ddd`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:27.694851 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3284f-86c6-44c2-a9e5-917d00250451 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:27.694993 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-5c7b7925-ecc4-4b0a-824d-be00966f3475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:27.699475 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3284f-86c6-44c2-a9e5-917d00250451 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:27.721497 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:27.728087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 655/1313] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:27.739244 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81139ba-3209-4709-985c-f81235a925b8 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:27.739633 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81139ba-3209-4709-985c-f81235a925b8 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:27.739812 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81139ba-3209-4709-985c-f81235a925b8 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:27.745373 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 656/1314] 10.222.0.23 () {54 vars in 912 bytes} [Thu Oct 20 13:35:27 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:35:27.749649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c03d54-bf85-468c-8ef4-484683b712e8 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:27.749953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c03d54-bf85-468c-8ef4-484683b712e8 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:27.750230 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c03d54-bf85-468c-8ef4-484683b712e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:27.819622 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-98c03d54-bf85-468c-8ef4-484683b712e8 None None] MFA Rules not processed for user `ea511efaebcb4b03aaf726688e230291`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:27.821029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-98c03d54-bf85-468c-8ef4-484683b712e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:27.837335 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 659/1315] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:27 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:27.839163 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30974c11-25ec-44c9-8553-ba1eab8832b8 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:27.839627 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30974c11-25ec-44c9-8553-ba1eab8832b8 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:27.840116 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30974c11-25ec-44c9-8553-ba1eab8832b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:27.937146 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-30974c11-25ec-44c9-8553-ba1eab8832b8 None None] MFA Rules not processed for user `ee5c2a3855cb4a668177aa19f337778d`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:27.940048 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-30974c11-25ec-44c9-8553-ba1eab8832b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:27.998942 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 657/1316] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:27 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:27.999695 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c03d54-bf85-468c-8ef4-484683b712e8 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:28.110530 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c03d54-bf85-468c-8ef4-484683b712e8 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:28.115957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a46a2-dd42-45f1-a94c-365e8d2d8eb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:28.116794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a46a2-dd42-45f1-a94c-365e8d2d8eb9 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:28.117206 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a46a2-dd42-45f1-a94c-365e8d2d8eb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:28.117675 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a46a2-dd42-45f1-a94c-365e8d2d8eb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:28.151253 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a46a2-dd42-45f1-a94c-365e8d2d8eb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:28.156768 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a46a2-dd42-45f1-a94c-365e8d2d8eb9 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:28.175915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 660/1317] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 20 13:35:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:28.176936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30974c11-25ec-44c9-8553-ba1eab8832b8 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:28.179355 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 658/1318] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:27 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:28.180345 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a46a2-dd42-45f1-a94c-365e8d2d8eb9 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:28.204644 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a46a2-dd42-45f1-a94c-365e8d2d8eb9 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:28.214528 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0b396-a323-40c9-a474-fa92dccce886 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:28.216464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0b396-a323-40c9-a474-fa92dccce886 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:28.217009 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0b396-a323-40c9-a474-fa92dccce886 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:28.217549 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0b396-a323-40c9-a474-fa92dccce886 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:28.254036 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0b396-a323-40c9-a474-fa92dccce886 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:28.257798 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0b396-a323-40c9-a474-fa92dccce886 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:28.298466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 659/1319] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:28 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:28.300617 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0b396-a323-40c9-a474-fa92dccce886 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:28.322037 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0b396-a323-40c9-a474-fa92dccce886 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:28.339553 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb50c2-97c7-4854-bc7e-693e925b9ebd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:28.340371 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb50c2-97c7-4854-bc7e-693e925b9ebd service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:28.340739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb50c2-97c7-4854-bc7e-693e925b9ebd service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:28.341094 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb50c2-97c7-4854-bc7e-693e925b9ebd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:28.344222 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30974c11-25ec-44c9-8553-ba1eab8832b8 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:28.352668 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f6a2d-8f2e-445c-afa4-f91d3dd4989e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:28.354042 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f6a2d-8f2e-445c-afa4-f91d3dd4989e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:28.354330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f6a2d-8f2e-445c-afa4-f91d3dd4989e admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:28.354663 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f6a2d-8f2e-445c-afa4-f91d3dd4989e admin admin] PATH_INFO: `/v3/users/47ea84f99dc2419581cf8fa21e26f823` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:28.355931 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f6a2d-8f2e-445c-afa4-f91d3dd4989e admin admin] RBAC: Authorizing `identity:delete_user(user_id=47ea84f99dc2419581cf8fa21e26f823)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:28.357467 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: WARNING py.warnings [None req-3b4f6a2d-8f2e-445c-afa4-f91d3dd4989e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 13:35:28.357467 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: warnings.warn(msg) Oct 20 13:35:28.357467 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: Oct 20 13:35:28.359161 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f6a2d-8f2e-445c-afa4-f91d3dd4989e admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:28.364244 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccb50c2-97c7-4854-bc7e-693e925b9ebd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:28.366357 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccb50c2-97c7-4854-bc7e-693e925b9ebd service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:28.410954 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 660/1320] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:28.473631 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-3b4f6a2d-8f2e-445c-afa4-f91d3dd4989e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47ea84f99dc2419581cf8fa21e26f823'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:28.475954 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-3b4f6a2d-8f2e-445c-afa4-f91d3dd4989e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47ea84f99dc2419581cf8fa21e26f823'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:28.479845 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:28.503069 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-3b4f6a2d-8f2e-445c-afa4-f91d3dd4989e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47ea84f99dc2419581cf8fa21e26f823'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:28.510795 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-3b4f6a2d-8f2e-445c-afa4-f91d3dd4989e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47ea84f99dc2419581cf8fa21e26f823'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:28.512220 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb50c2-97c7-4854-bc7e-693e925b9ebd service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:28.522478 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb50c2-97c7-4854-bc7e-693e925b9ebd service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:28.526755 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da0180-8ebc-4448-90ed-634edc1f203a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:28.527453 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da0180-8ebc-4448-90ed-634edc1f203a service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:28.527937 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da0180-8ebc-4448-90ed-634edc1f203a service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:28.538652 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-3b4f6a2d-8f2e-445c-afa4-f91d3dd4989e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47ea84f99dc2419581cf8fa21e26f823'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:28.543383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da0180-8ebc-4448-90ed-634edc1f203a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:28.550952 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c0da0180-8ebc-4448-90ed-634edc1f203a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:28.593183 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 661/1321] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:35:28 2022] DELETE /identity/v3/users/47ea84f99dc2419581cf8fa21e26f823 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:28.597731 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31880437-ff1e-4448-ab56-7f31c6a7ec4f None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:28.598139 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31880437-ff1e-4448-ab56-7f31c6a7ec4f None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:28.599210 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31880437-ff1e-4448-ab56-7f31c6a7ec4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:28.603296 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:28.623724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-31880437-ff1e-4448-ab56-7f31c6a7ec4f None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:28.624996 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-31880437-ff1e-4448-ab56-7f31c6a7ec4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:28.672205 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da0180-8ebc-4448-90ed-634edc1f203a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:28.674993 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da0180-8ebc-4448-90ed-634edc1f203a service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:28.692459 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 661/1322] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:28 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:28.786133 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 662/1323] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:35:28.804131 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da0180-8ebc-4448-90ed-634edc1f203a service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:28.805570 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c0da0180-8ebc-4448-90ed-634edc1f203a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:28.970119 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da0180-8ebc-4448-90ed-634edc1f203a service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:29.072658 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d176d-d69e-4039-a04d-309640c2a55b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:29.073450 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d176d-d69e-4039-a04d-309640c2a55b service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:29.073835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d176d-d69e-4039-a04d-309640c2a55b service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:29.074262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d176d-d69e-4039-a04d-309640c2a55b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:29.102715 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d176d-d69e-4039-a04d-309640c2a55b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:29.104501 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d176d-d69e-4039-a04d-309640c2a55b service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:29.125011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 662/1324] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:29.402423 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31880437-ff1e-4448-ab56-7f31c6a7ec4f None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:29.404550 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-31880437-ff1e-4448-ab56-7f31c6a7ec4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:29.424257 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31880437-ff1e-4448-ab56-7f31c6a7ec4f None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:29.435309 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1a4d8-e840-45b5-8e9c-46e16d1c644f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:29.436282 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1a4d8-e840-45b5-8e9c-46e16d1c644f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:29.436282 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1a4d8-e840-45b5-8e9c-46e16d1c644f admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:29.436499 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1a4d8-e840-45b5-8e9c-46e16d1c644f admin admin] PATH_INFO: `/v3/users/dbfb17284ffe448cb2e2c0713123464b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:29.438835 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b1a4d8-e840-45b5-8e9c-46e16d1c644f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbfb17284ffe448cb2e2c0713123464b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:29.442827 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b1a4d8-e840-45b5-8e9c-46e16d1c644f admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:29.555199 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c2b1a4d8-e840-45b5-8e9c-46e16d1c644f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbfb17284ffe448cb2e2c0713123464b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:29.557379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c2b1a4d8-e840-45b5-8e9c-46e16d1c644f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbfb17284ffe448cb2e2c0713123464b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:29.576507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c2b1a4d8-e840-45b5-8e9c-46e16d1c644f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbfb17284ffe448cb2e2c0713123464b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:29.576779 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c2b1a4d8-e840-45b5-8e9c-46e16d1c644f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbfb17284ffe448cb2e2c0713123464b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:29.597000 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c2b1a4d8-e840-45b5-8e9c-46e16d1c644f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbfb17284ffe448cb2e2c0713123464b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:29.645934 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 663/1325] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:35:29 2022] DELETE /identity/v3/users/dbfb17284ffe448cb2e2c0713123464b => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:29.651014 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5ad33-1c53-477e-84fc-874bfacb4ea0 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:29.651308 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5ad33-1c53-477e-84fc-874bfacb4ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:29.651592 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5ad33-1c53-477e-84fc-874bfacb4ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:29.685774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-79a5ad33-1c53-477e-84fc-874bfacb4ea0 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:29.687531 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-79a5ad33-1c53-477e-84fc-874bfacb4ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:29.808276 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 663/1326] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:29.884365 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1a4d8-e840-45b5-8e9c-46e16d1c644f admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:29.885956 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-c2b1a4d8-e840-45b5-8e9c-46e16d1c644f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:29.963407 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:30.007048 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1a4d8-e840-45b5-8e9c-46e16d1c644f admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:30.014124 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c049c-ddfb-4a31-a732-65d44fc0b556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:30.015067 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c049c-ddfb-4a31-a732-65d44fc0b556 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:30.015446 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c049c-ddfb-4a31-a732-65d44fc0b556 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:30.016444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c049c-ddfb-4a31-a732-65d44fc0b556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:30.029883 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0c049c-ddfb-4a31-a732-65d44fc0b556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:30.031928 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0c049c-ddfb-4a31-a732-65d44fc0b556 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:30.062269 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 664/1327] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:31.085452 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5ad33-1c53-477e-84fc-874bfacb4ea0 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:31.085452 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-79a5ad33-1c53-477e-84fc-874bfacb4ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:31.085452 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5ad33-1c53-477e-84fc-874bfacb4ea0 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:31.087739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10261052-ab90-48b5-9233-d2fb5f6e99c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:31.087739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10261052-ab90-48b5-9233-d2fb5f6e99c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:31.089867 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10261052-ab90-48b5-9233-d2fb5f6e99c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:31.092283 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10261052-ab90-48b5-9233-d2fb5f6e99c9 admin admin] PATH_INFO: `/v3/projects/4f8ffcea74224edc99bd7c9fd4b796ec` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:31.096262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10261052-ab90-48b5-9233-d2fb5f6e99c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f8ffcea74224edc99bd7c9fd4b796ec)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:31.100683 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: WARNING py.warnings [None req-10261052-ab90-48b5-9233-d2fb5f6e99c9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 20 13:35:31.100683 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: warnings.warn(msg) Oct 20 13:35:31.100683 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: Oct 20 13:35:31.105550 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10261052-ab90-48b5-9233-d2fb5f6e99c9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:31.239605 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:31.323548 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-10261052-ab90-48b5-9233-d2fb5f6e99c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f8ffcea74224edc99bd7c9fd4b796ec', 'request_id': 'req-10261052-ab90-48b5-9233-d2fb5f6e99c9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:31.341183 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c049c-ddfb-4a31-a732-65d44fc0b556 service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:31.342878 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-db0c049c-ddfb-4a31-a732-65d44fc0b556 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:31.362806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-10261052-ab90-48b5-9233-d2fb5f6e99c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f8ffcea74224edc99bd7c9fd4b796ec', 'request_id': 'req-10261052-ab90-48b5-9233-d2fb5f6e99c9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:31.388188 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-10261052-ab90-48b5-9233-d2fb5f6e99c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f8ffcea74224edc99bd7c9fd4b796ec', 'request_id': 'req-10261052-ab90-48b5-9233-d2fb5f6e99c9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:31.393432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c049c-ddfb-4a31-a732-65d44fc0b556 service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:31.434330 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-10261052-ab90-48b5-9233-d2fb5f6e99c9 admin admin] The token cache is being invalidate because project 4f8ffcea74224edc99bd7c9fd4b796ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:35:31.434801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-10261052-ab90-48b5-9233-d2fb5f6e99c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:31.436391 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-10261052-ab90-48b5-9233-d2fb5f6e99c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:31.441711 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 664/1328] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:35:31 2022] DELETE /identity/v3/projects/4f8ffcea74224edc99bd7c9fd4b796ec => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:31.447863 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10261052-ab90-48b5-9233-d2fb5f6e99c9 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:31.450410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-10261052-ab90-48b5-9233-d2fb5f6e99c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:31.542917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab00d2-ba3e-4629-9b5c-244a0ba719e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:31.543974 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab00d2-ba3e-4629-9b5c-244a0ba719e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:31.544464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab00d2-ba3e-4629-9b5c-244a0ba719e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:31.544942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab00d2-ba3e-4629-9b5c-244a0ba719e3 admin admin] PATH_INFO: `/v3/projects/3d4302a426b84e7696c67066b62c1fee` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:31.546534 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab00d2-ba3e-4629-9b5c-244a0ba719e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d4302a426b84e7696c67066b62c1fee)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:31.550885 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab00d2-ba3e-4629-9b5c-244a0ba719e3 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:31.579068 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10261052-ab90-48b5-9233-d2fb5f6e99c9 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:31.585348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ee0a8-74ad-4dde-a87c-96b97f37a18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:31.586161 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ee0a8-74ad-4dde-a87c-96b97f37a18a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:31.586593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ee0a8-74ad-4dde-a87c-96b97f37a18a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:31.587273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ee0a8-74ad-4dde-a87c-96b97f37a18a admin admin] PATH_INFO: `/v3/users/ea511efaebcb4b03aaf726688e230291` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:31.589252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ee0a8-74ad-4dde-a87c-96b97f37a18a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea511efaebcb4b03aaf726688e230291)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:31.592018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ee0a8-74ad-4dde-a87c-96b97f37a18a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:31.666365 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-86ab00d2-ba3e-4629-9b5c-244a0ba719e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d4302a426b84e7696c67066b62c1fee', 'request_id': 'req-86ab00d2-ba3e-4629-9b5c-244a0ba719e3'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:31.667415 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a45ee0a8-74ad-4dde-a87c-96b97f37a18a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea511efaebcb4b03aaf726688e230291'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:31.670071 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a45ee0a8-74ad-4dde-a87c-96b97f37a18a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea511efaebcb4b03aaf726688e230291'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:31.674404 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-86ab00d2-ba3e-4629-9b5c-244a0ba719e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d4302a426b84e7696c67066b62c1fee', 'request_id': 'req-86ab00d2-ba3e-4629-9b5c-244a0ba719e3'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:31.688575 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a45ee0a8-74ad-4dde-a87c-96b97f37a18a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea511efaebcb4b03aaf726688e230291'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:31.692621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-86ab00d2-ba3e-4629-9b5c-244a0ba719e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d4302a426b84e7696c67066b62c1fee', 'request_id': 'req-86ab00d2-ba3e-4629-9b5c-244a0ba719e3'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:31.718100 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a45ee0a8-74ad-4dde-a87c-96b97f37a18a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea511efaebcb4b03aaf726688e230291'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:31.718763 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a45ee0a8-74ad-4dde-a87c-96b97f37a18a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea511efaebcb4b03aaf726688e230291'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:31.725400 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-86ab00d2-ba3e-4629-9b5c-244a0ba719e3 admin admin] The token cache is being invalidate because project 3d4302a426b84e7696c67066b62c1fee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:35:31.725719 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-86ab00d2-ba3e-4629-9b5c-244a0ba719e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:31.726566 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-86ab00d2-ba3e-4629-9b5c-244a0ba719e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:31.734257 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 665/1329] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:35:31 2022] DELETE /identity/v3/projects/3d4302a426b84e7696c67066b62c1fee => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:35:31.739660 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ca8f2-a0ff-4c8b-84f7-397fa8431530 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:31.740504 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ca8f2-a0ff-4c8b-84f7-397fa8431530 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:31.741016 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ca8f2-a0ff-4c8b-84f7-397fa8431530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:31.766362 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 665/1330] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:35:31 2022] DELETE /identity/v3/users/ea511efaebcb4b03aaf726688e230291 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:31.780227 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-9d3ca8f2-a0ff-4c8b-84f7-397fa8431530 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:31.783341 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-9d3ca8f2-a0ff-4c8b-84f7-397fa8431530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:31.949065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 666/1331] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:31.956745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d1bd6-58d6-4ac2-9eae-263803975f6c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:31.957092 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d1bd6-58d6-4ac2-9eae-263803975f6c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:31.957379 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d1bd6-58d6-4ac2-9eae-263803975f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:31.982147 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d96d1bd6-58d6-4ac2-9eae-263803975f6c None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:31.983566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d96d1bd6-58d6-4ac2-9eae-263803975f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:31.993540 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 666/1332] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:31.999546 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ca8f2-a0ff-4c8b-84f7-397fa8431530 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:32.012191 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ca8f2-a0ff-4c8b-84f7-397fa8431530 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:32.036680 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c07a63-54fa-463f-84c0-731298d2cb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:32.038487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c07a63-54fa-463f-84c0-731298d2cb48 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:32.038937 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c07a63-54fa-463f-84c0-731298d2cb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:32.039365 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c07a63-54fa-463f-84c0-731298d2cb48 admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:32.040212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c07a63-54fa-463f-84c0-731298d2cb48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:35:32.040674 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c07a63-54fa-463f-84c0-731298d2cb48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:32.042717 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c07a63-54fa-463f-84c0-731298d2cb48 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:32.062650 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:32.069245 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 667/1333] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:35:32.073365 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d1bd6-58d6-4ac2-9eae-263803975f6c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:32.102009 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d1bd6-58d6-4ac2-9eae-263803975f6c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:32.111103 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b822345-18c0-4c3a-89c0-5c3fd0d26944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:32.112148 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b822345-18c0-4c3a-89c0-5c3fd0d26944 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:32.112583 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b822345-18c0-4c3a-89c0-5c3fd0d26944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:32.113046 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b822345-18c0-4c3a-89c0-5c3fd0d26944 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:32.114235 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b822345-18c0-4c3a-89c0-5c3fd0d26944 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:32.117282 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b822345-18c0-4c3a-89c0-5c3fd0d26944 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:32.190799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c07a63-54fa-463f-84c0-731298d2cb48 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:32.199299 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 667/1334] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:35:32 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:32.205181 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c07a63-54fa-463f-84c0-731298d2cb48 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:32.310228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b4090-2762-4ea1-83fe-9eab12e97123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:32.311073 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b4090-2762-4ea1-83fe-9eab12e97123 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:32.311491 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b4090-2762-4ea1-83fe-9eab12e97123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:32.311843 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b4090-2762-4ea1-83fe-9eab12e97123 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:32.312570 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b4090-2762-4ea1-83fe-9eab12e97123 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:32.314843 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b4090-2762-4ea1-83fe-9eab12e97123 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:32.367233 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 668/1335] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:35:32 2022] POST /identity/v3/users => generated 287 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:32.367732 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b822345-18c0-4c3a-89c0-5c3fd0d26944 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:32.375242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:32.384027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b822345-18c0-4c3a-89c0-5c3fd0d26944 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:32.418611 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f648f-a9be-4474-9130-b240e2d894a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:32.419679 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f648f-a9be-4474-9130-b240e2d894a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:32.422447 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f648f-a9be-4474-9130-b240e2d894a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:32.425448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f648f-a9be-4474-9130-b240e2d894a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:32.426628 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f648f-a9be-4474-9130-b240e2d894a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:32.431881 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f648f-a9be-4474-9130-b240e2d894a7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:32.460239 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 668/1336] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:32.464409 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b4090-2762-4ea1-83fe-9eab12e97123 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:32.482609 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b4090-2762-4ea1-83fe-9eab12e97123 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:32.499113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e3e80-f899-40f8-8662-4eb0ed946510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:32.499113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e3e80-f899-40f8-8662-4eb0ed946510 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:32.499113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e3e80-f899-40f8-8662-4eb0ed946510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:32.499113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e3e80-f899-40f8-8662-4eb0ed946510 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:32.499585 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e3e80-f899-40f8-8662-4eb0ed946510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:32.500532 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e3e80-f899-40f8-8662-4eb0ed946510 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:32.528542 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 669/1337] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:32.532412 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f648f-a9be-4474-9130-b240e2d894a7 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:32.558714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f648f-a9be-4474-9130-b240e2d894a7 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:32.574808 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7922f-b4c3-4c4a-ae6f-560f6df15a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:32.575660 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7922f-b4c3-4c4a-ae6f-560f6df15a96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:32.575946 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7922f-b4c3-4c4a-ae6f-560f6df15a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:32.576223 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7922f-b4c3-4c4a-ae6f-560f6df15a96 admin admin] PATH_INFO: `/v3/projects/db84fe89e0934db69643734c541c9824/users/dd2e24f606434ede8b6e9d68312d144c/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:32.607397 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7922f-b4c3-4c4a-ae6f-560f6df15a96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db84fe89e0934db69643734c541c9824, user_id=dd2e24f606434ede8b6e9d68312d144c, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:32.610420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7922f-b4c3-4c4a-ae6f-560f6df15a96 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:32.653422 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 669/1338] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:32 2022] PUT /identity/v3/projects/db84fe89e0934db69643734c541c9824/users/dd2e24f606434ede8b6e9d68312d144c/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:32.670847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20dc94-9656-4beb-8016-0ab15336f5e7 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:32.671140 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20dc94-9656-4beb-8016-0ab15336f5e7 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:32.671423 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20dc94-9656-4beb-8016-0ab15336f5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:32.756213 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-2c20dc94-9656-4beb-8016-0ab15336f5e7 None None] MFA Rules not processed for user `dd2e24f606434ede8b6e9d68312d144c`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:32.759053 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-2c20dc94-9656-4beb-8016-0ab15336f5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:32.906380 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 670/1339] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:32.907292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7922f-b4c3-4c4a-ae6f-560f6df15a96 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:33.066950 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7922f-b4c3-4c4a-ae6f-560f6df15a96 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:33.081745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeca393-b0c0-48eb-a589-74c4783912cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:33.082659 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeca393-b0c0-48eb-a589-74c4783912cc admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:33.082930 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeca393-b0c0-48eb-a589-74c4783912cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:33.083232 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeca393-b0c0-48eb-a589-74c4783912cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:33.086606 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeca393-b0c0-48eb-a589-74c4783912cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:33.091290 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeca393-b0c0-48eb-a589-74c4783912cc admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:33.161608 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 670/1340] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:35:32 2022] POST /identity/v3/projects => generated 371 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:35:33.165558 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20dc94-9656-4beb-8016-0ab15336f5e7 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:33.184790 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20dc94-9656-4beb-8016-0ab15336f5e7 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:33.378002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348c021-b30a-453c-aec9-51afb6228803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:33.379832 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348c021-b30a-453c-aec9-51afb6228803 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:33.380466 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348c021-b30a-453c-aec9-51afb6228803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:33.381078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348c021-b30a-453c-aec9-51afb6228803 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:33.381896 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a348c021-b30a-453c-aec9-51afb6228803 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:33.385644 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a348c021-b30a-453c-aec9-51afb6228803 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:33.425399 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:33.439565 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 671/1341] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:35:33 2022] POST /identity/v3/users => generated 292 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:33.440801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeca393-b0c0-48eb-a589-74c4783912cc admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:33.466336 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeca393-b0c0-48eb-a589-74c4783912cc admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:33.481251 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1099605-6982-4dc5-9231-1feb9068f9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:33.483087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1099605-6982-4dc5-9231-1feb9068f9a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:33.484538 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1099605-6982-4dc5-9231-1feb9068f9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:33.484751 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1099605-6982-4dc5-9231-1feb9068f9a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:33.486803 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1099605-6982-4dc5-9231-1feb9068f9a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:33.491965 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1099605-6982-4dc5-9231-1feb9068f9a1 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:33.549527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 671/1342] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:35:33.550598 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348c021-b30a-453c-aec9-51afb6228803 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:33.565660 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348c021-b30a-453c-aec9-51afb6228803 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:33.585059 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d233c0a9-0253-4216-9b07-04448fe1c5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:33.585999 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233c0a9-0253-4216-9b07-04448fe1c5c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:33.586757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233c0a9-0253-4216-9b07-04448fe1c5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:33.587806 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233c0a9-0253-4216-9b07-04448fe1c5c6 admin admin] PATH_INFO: `/v3/projects/5ac3e8306f2946b3aaa0978bf6337e04/users/7617b4ac9cc24217865b05b2837a2c1b/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:33.669277 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d233c0a9-0253-4216-9b07-04448fe1c5c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac3e8306f2946b3aaa0978bf6337e04, user_id=7617b4ac9cc24217865b05b2837a2c1b, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:33.695692 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d233c0a9-0253-4216-9b07-04448fe1c5c6 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:33.731660 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 672/1343] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:33 2022] PUT /identity/v3/projects/5ac3e8306f2946b3aaa0978bf6337e04/users/7617b4ac9cc24217865b05b2837a2c1b/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:33.733012 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1099605-6982-4dc5-9231-1feb9068f9a1 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:33.745624 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1099605-6982-4dc5-9231-1feb9068f9a1 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:33.904303 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24325e-01a2-4798-89a1-7c9804ed97a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:33.905041 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24325e-01a2-4798-89a1-7c9804ed97a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:33.905335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24325e-01a2-4798-89a1-7c9804ed97a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:33.905599 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24325e-01a2-4798-89a1-7c9804ed97a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:33.906253 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24325e-01a2-4798-89a1-7c9804ed97a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:33.907879 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24325e-01a2-4798-89a1-7c9804ed97a9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:33.946172 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 672/1344] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:33.950969 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d233c0a9-0253-4216-9b07-04448fe1c5c6 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:33.965962 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d233c0a9-0253-4216-9b07-04448fe1c5c6 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:33.976817 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd543609-3518-41c6-8bd2-8bf2aa160cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:33.977531 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd543609-3518-41c6-8bd2-8bf2aa160cf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:33.977808 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd543609-3518-41c6-8bd2-8bf2aa160cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:33.978081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd543609-3518-41c6-8bd2-8bf2aa160cf9 admin admin] PATH_INFO: `/v3/projects/5ac3e8306f2946b3aaa0978bf6337e04/users/7617b4ac9cc24217865b05b2837a2c1b/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:33.986512 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd543609-3518-41c6-8bd2-8bf2aa160cf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac3e8306f2946b3aaa0978bf6337e04, user_id=7617b4ac9cc24217865b05b2837a2c1b, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:33.995517 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd543609-3518-41c6-8bd2-8bf2aa160cf9 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:34.011949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 673/1345] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:33 2022] PUT /identity/v3/projects/5ac3e8306f2946b3aaa0978bf6337e04/users/7617b4ac9cc24217865b05b2837a2c1b/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:34.023545 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bb328-b3d9-4cc3-9ac0-9171845772e9 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:34.024651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bb328-b3d9-4cc3-9ac0-9171845772e9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:34.025086 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bb328-b3d9-4cc3-9ac0-9171845772e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:34.110296 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f72bb328-b3d9-4cc3-9ac0-9171845772e9 None None] MFA Rules not processed for user `7617b4ac9cc24217865b05b2837a2c1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:34.111651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f72bb328-b3d9-4cc3-9ac0-9171845772e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:34.203161 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:34.262367 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 673/1346] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:34.322780 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd543609-3518-41c6-8bd2-8bf2aa160cf9 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:34.324534 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-cd543609-3518-41c6-8bd2-8bf2aa160cf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:34.523547 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd543609-3518-41c6-8bd2-8bf2aa160cf9 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:34.531032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72bb328-b3d9-4cc3-9ac0-9171845772e9 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:34.533149 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f72bb328-b3d9-4cc3-9ac0-9171845772e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:34.539178 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34f518-5f05-48c4-945e-94abb44226e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:34.539822 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34f518-5f05-48c4-945e-94abb44226e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:34.540108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34f518-5f05-48c4-945e-94abb44226e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:34.540389 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34f518-5f05-48c4-945e-94abb44226e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:34.580185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf34f518-5f05-48c4-945e-94abb44226e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:34.587792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf34f518-5f05-48c4-945e-94abb44226e1 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:34.617246 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 674/1347] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:34.769273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72bb328-b3d9-4cc3-9ac0-9171845772e9 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:34.791758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d851aa5-9eb1-4e54-bbf9-e2d6f7fb50ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:34.794266 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d851aa5-9eb1-4e54-bbf9-e2d6f7fb50ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:34.794742 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d851aa5-9eb1-4e54-bbf9-e2d6f7fb50ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:34.797974 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d851aa5-9eb1-4e54-bbf9-e2d6f7fb50ed admin admin] PATH_INFO: `/v3/projects/e29cab0a4c194b63a9ae234afd9256cd` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:34.801919 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d851aa5-9eb1-4e54-bbf9-e2d6f7fb50ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=e29cab0a4c194b63a9ae234afd9256cd)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:34.809377 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d851aa5-9eb1-4e54-bbf9-e2d6f7fb50ed admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:34.962780 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3d851aa5-9eb1-4e54-bbf9-e2d6f7fb50ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e29cab0a4c194b63a9ae234afd9256cd', 'request_id': 'req-3d851aa5-9eb1-4e54-bbf9-e2d6f7fb50ed'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:34.984589 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34f518-5f05-48c4-945e-94abb44226e1 service cinder] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:34.986456 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-cf34f518-5f05-48c4-945e-94abb44226e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:34.991033 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3d851aa5-9eb1-4e54-bbf9-e2d6f7fb50ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e29cab0a4c194b63a9ae234afd9256cd', 'request_id': 'req-3d851aa5-9eb1-4e54-bbf9-e2d6f7fb50ed'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:35.013740 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34f518-5f05-48c4-945e-94abb44226e1 service cinder] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:35.023674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3d851aa5-9eb1-4e54-bbf9-e2d6f7fb50ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e29cab0a4c194b63a9ae234afd9256cd', 'request_id': 'req-3d851aa5-9eb1-4e54-bbf9-e2d6f7fb50ed'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:35.073008 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3d851aa5-9eb1-4e54-bbf9-e2d6f7fb50ed admin admin] The token cache is being invalidate because project e29cab0a4c194b63a9ae234afd9256cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:35:35.073489 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3d851aa5-9eb1-4e54-bbf9-e2d6f7fb50ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:35.075046 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3d851aa5-9eb1-4e54-bbf9-e2d6f7fb50ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:35.079152 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 674/1348] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:35:34 2022] DELETE /identity/v3/projects/e29cab0a4c194b63a9ae234afd9256cd => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:35.084713 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa84bfb-930a-454d-939c-33987385461b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:35.085161 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa84bfb-930a-454d-939c-33987385461b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:35.087318 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa84bfb-930a-454d-939c-33987385461b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:35.152790 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f93b10-4203-43aa-a1e7-3a491be5d9c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:35.154191 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f93b10-4203-43aa-a1e7-3a491be5d9c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:35.155411 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f93b10-4203-43aa-a1e7-3a491be5d9c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:35.155851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f93b10-4203-43aa-a1e7-3a491be5d9c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:35.158622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-85f93b10-4203-43aa-a1e7-3a491be5d9c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:35.170772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2fa84bfb-930a-454d-939c-33987385461b None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:35.176546 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2fa84bfb-930a-454d-939c-33987385461b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:35.329939 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f93b10-4203-43aa-a1e7-3a491be5d9c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:35.332210 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f93b10-4203-43aa-a1e7-3a491be5d9c0 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:35.370596 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 675/1349] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:35:35.397158 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:35.436032 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f93b10-4203-43aa-a1e7-3a491be5d9c0 service cinder] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:35.440330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-85f93b10-4203-43aa-a1e7-3a491be5d9c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:35.472673 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 675/1350] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:35.478245 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dfe7a-b657-4ff3-87bc-f6eb87a67681 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:35.478721 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dfe7a-b657-4ff3-87bc-f6eb87a67681 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:35.479173 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dfe7a-b657-4ff3-87bc-f6eb87a67681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:35.494555 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:35.534177 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f22dfe7a-b657-4ff3-87bc-f6eb87a67681 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:35.535678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f22dfe7a-b657-4ff3-87bc-f6eb87a67681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:35.561215 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 676/1351] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:35.566871 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dfe7a-b657-4ff3-87bc-f6eb87a67681 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:35.605825 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dfe7a-b657-4ff3-87bc-f6eb87a67681 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:35.623292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45682bb3-6fb8-4877-b329-77d6310477bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:35.624124 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45682bb3-6fb8-4877-b329-77d6310477bf admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:35.624634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45682bb3-6fb8-4877-b329-77d6310477bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:35.625110 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45682bb3-6fb8-4877-b329-77d6310477bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:35.625941 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45682bb3-6fb8-4877-b329-77d6310477bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:35:35.626386 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45682bb3-6fb8-4877-b329-77d6310477bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:35.628300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45682bb3-6fb8-4877-b329-77d6310477bf admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:35.665141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 677/1352] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:35:35.666150 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45682bb3-6fb8-4877-b329-77d6310477bf admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:35.689972 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45682bb3-6fb8-4877-b329-77d6310477bf admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:35.712784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc322c-416c-4cfb-a5de-52733db762ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:35.715840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc322c-416c-4cfb-a5de-52733db762ec admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:35.716140 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc322c-416c-4cfb-a5de-52733db762ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:35.717421 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc322c-416c-4cfb-a5de-52733db762ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:35.717698 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc322c-416c-4cfb-a5de-52733db762ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:35.729801 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f93b10-4203-43aa-a1e7-3a491be5d9c0 service cinder] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:35.731500 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc322c-416c-4cfb-a5de-52733db762ec admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:35.759661 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9e3e6-89d1-4c78-bae3-7c91c1f8dbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:35.768004 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9e3e6-89d1-4c78-bae3-7c91c1f8dbe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:35.772252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9e3e6-89d1-4c78-bae3-7c91c1f8dbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:35.772801 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9e3e6-89d1-4c78-bae3-7c91c1f8dbe8 admin admin] PATH_INFO: `/v3/users/a49d0c317d3d4e4aa7b98e5167061c11` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:35.782966 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9e3e6-89d1-4c78-bae3-7c91c1f8dbe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a49d0c317d3d4e4aa7b98e5167061c11)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:35.807586 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9e3e6-89d1-4c78-bae3-7c91c1f8dbe8 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:35.814990 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 678/1353] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:35:35 2022] POST /identity/v3/projects => generated 369 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:35.818861 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc322c-416c-4cfb-a5de-52733db762ec admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:35.967811 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc322c-416c-4cfb-a5de-52733db762ec admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:35.977029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e6933-638c-41f8-9711-345957bc95b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:35.978480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e6933-638c-41f8-9711-345957bc95b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:35.979100 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e6933-638c-41f8-9711-345957bc95b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:35.979757 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e6933-638c-41f8-9711-345957bc95b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:35.987565 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e6933-638c-41f8-9711-345957bc95b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:35.988270 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-3dd9e3e6-89d1-4c78-bae3-7c91c1f8dbe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a49d0c317d3d4e4aa7b98e5167061c11'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:35.989178 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-3dd9e3e6-89d1-4c78-bae3-7c91c1f8dbe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a49d0c317d3d4e4aa7b98e5167061c11'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:35.991683 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e6933-638c-41f8-9711-345957bc95b3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:36.009178 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-3dd9e3e6-89d1-4c78-bae3-7c91c1f8dbe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a49d0c317d3d4e4aa7b98e5167061c11'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:36.011703 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-3dd9e3e6-89d1-4c78-bae3-7c91c1f8dbe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a49d0c317d3d4e4aa7b98e5167061c11'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:36.029810 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 679/1354] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:35:35 2022] POST /identity/v3/users => generated 285 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:36.031699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-3dd9e3e6-89d1-4c78-bae3-7c91c1f8dbe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a49d0c317d3d4e4aa7b98e5167061c11'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:36.033143 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e6933-638c-41f8-9711-345957bc95b3 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:36.035277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2d7e6933-638c-41f8-9711-345957bc95b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:36.065386 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e6933-638c-41f8-9711-345957bc95b3 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:36.081105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 676/1355] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:35:35 2022] DELETE /identity/v3/users/a49d0c317d3d4e4aa7b98e5167061c11 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:35:36.082702 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50513754-960a-4d74-be97-b47da19f4008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:36.083790 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50513754-960a-4d74-be97-b47da19f4008 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:36.084018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50513754-960a-4d74-be97-b47da19f4008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:36.084334 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50513754-960a-4d74-be97-b47da19f4008 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:36.085080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50513754-960a-4d74-be97-b47da19f4008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:36.085869 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71f892-8e67-4928-a6b8-fad231597c23 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:36.086159 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71f892-8e67-4928-a6b8-fad231597c23 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:36.086777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50513754-960a-4d74-be97-b47da19f4008 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:36.086927 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71f892-8e67-4928-a6b8-fad231597c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:36.115829 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 680/1356] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:35:36.118556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50513754-960a-4d74-be97-b47da19f4008 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:36.136465 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50513754-960a-4d74-be97-b47da19f4008 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:36.149653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-7d71f892-8e67-4928-a6b8-fad231597c23 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:36.155833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-7d71f892-8e67-4928-a6b8-fad231597c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:36.294540 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 677/1357] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:36.300745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84679c-3be0-4639-9822-f51b0e09364d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:36.301610 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84679c-3be0-4639-9822-f51b0e09364d admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:36.302064 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84679c-3be0-4639-9822-f51b0e09364d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:36.303471 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84679c-3be0-4639-9822-f51b0e09364d admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:36.304276 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84679c-3be0-4639-9822-f51b0e09364d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:36.306094 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84679c-3be0-4639-9822-f51b0e09364d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:36.318242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 681/1358] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:36.322330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71f892-8e67-4928-a6b8-fad231597c23 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:36.339863 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84679c-3be0-4639-9822-f51b0e09364d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:36.342262 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71f892-8e67-4928-a6b8-fad231597c23 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:36.344132 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ac84679c-3be0-4639-9822-f51b0e09364d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:36.356944 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676e2ca-39d9-432b-94ec-862584636032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:36.358004 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676e2ca-39d9-432b-94ec-862584636032 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:36.358491 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676e2ca-39d9-432b-94ec-862584636032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:36.358925 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676e2ca-39d9-432b-94ec-862584636032 admin admin] PATH_INFO: `/v3/projects/8bf9e8d2f2394d39ab75f3b0dd031d76/users/584fc05e6c614e56a2d20ba78969a454/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:36.396796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676e2ca-39d9-432b-94ec-862584636032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bf9e8d2f2394d39ab75f3b0dd031d76, user_id=584fc05e6c614e56a2d20ba78969a454, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:36.400640 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676e2ca-39d9-432b-94ec-862584636032 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:36.421797 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:36.434552 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 678/1359] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:36 2022] PUT /identity/v3/projects/8bf9e8d2f2394d39ab75f3b0dd031d76/users/584fc05e6c614e56a2d20ba78969a454/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:36.446824 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962644b8-c644-47e2-8b9c-77ac76885563 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:36.447330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962644b8-c644-47e2-8b9c-77ac76885563 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:36.447771 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962644b8-c644-47e2-8b9c-77ac76885563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:36.453855 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84679c-3be0-4639-9822-f51b0e09364d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:36.511191 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:36.513472 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-962644b8-c644-47e2-8b9c-77ac76885563 None None] MFA Rules not processed for user `584fc05e6c614e56a2d20ba78969a454`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:36.515635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-962644b8-c644-47e2-8b9c-77ac76885563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:36.531979 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7876d9e-f436-4339-8598-d0510f670ef1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:36.532757 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7876d9e-f436-4339-8598-d0510f670ef1 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:36.533190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7876d9e-f436-4339-8598-d0510f670ef1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:36.533616 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7876d9e-f436-4339-8598-d0510f670ef1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:36.738218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7876d9e-f436-4339-8598-d0510f670ef1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:36.752663 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 679/1360] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:36.753344 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7876d9e-f436-4339-8598-d0510f670ef1 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:36.757711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962644b8-c644-47e2-8b9c-77ac76885563 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:36.800091 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962644b8-c644-47e2-8b9c-77ac76885563 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:36.801367 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 682/1361] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:36.812479 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4a0ac-4814-4800-bba3-d3003287f212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:36.813285 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4a0ac-4814-4800-bba3-d3003287f212 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:36.814112 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4a0ac-4814-4800-bba3-d3003287f212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:36.814388 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4a0ac-4814-4800-bba3-d3003287f212 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:36.815046 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4a0ac-4814-4800-bba3-d3003287f212 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:36.821697 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4a0ac-4814-4800-bba3-d3003287f212 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:36.886184 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 680/1362] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:35:36 2022] POST /identity/v3/projects => generated 367 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:36.889898 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7876d9e-f436-4339-8598-d0510f670ef1 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:36.905753 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7876d9e-f436-4339-8598-d0510f670ef1 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:37.162550 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fc80c-7a22-4219-87fe-c53d0d613132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:37.164246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fc80c-7a22-4219-87fe-c53d0d613132 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:37.164466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fc80c-7a22-4219-87fe-c53d0d613132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:37.165722 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fc80c-7a22-4219-87fe-c53d0d613132 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:37.167773 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7fc80c-7a22-4219-87fe-c53d0d613132 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:37.171232 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7fc80c-7a22-4219-87fe-c53d0d613132 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:37.274522 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4a0ac-4814-4800-bba3-d3003287f212 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:37.275670 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 683/1363] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:35:36 2022] POST /identity/v3/users => generated 290 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:35:37.292120 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4a0ac-4814-4800-bba3-d3003287f212 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:37.304663 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268c4d8-8149-4375-850f-c39d26cbc889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:37.305766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268c4d8-8149-4375-850f-c39d26cbc889 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:37.306316 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268c4d8-8149-4375-850f-c39d26cbc889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:37.306842 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268c4d8-8149-4375-850f-c39d26cbc889 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:37.307821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268c4d8-8149-4375-850f-c39d26cbc889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:37.309859 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268c4d8-8149-4375-850f-c39d26cbc889 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:37.329183 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fc80c-7a22-4219-87fe-c53d0d613132 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:37.336881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 681/1364] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:37.344257 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fc80c-7a22-4219-87fe-c53d0d613132 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:37.355425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca51300-3618-476b-89f3-80dca771f122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:37.357466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca51300-3618-476b-89f3-80dca771f122 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:37.358063 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca51300-3618-476b-89f3-80dca771f122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:37.359132 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca51300-3618-476b-89f3-80dca771f122 admin admin] PATH_INFO: `/v3/projects/740eca8d522c4a3ab6a691e7444f8f9d/users/7046bfe24636407f9a54c3663e2864fe/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:37.395519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca51300-3618-476b-89f3-80dca771f122 admin admin] RBAC: Authorizing `identity:create_grant(project_id=740eca8d522c4a3ab6a691e7444f8f9d, user_id=7046bfe24636407f9a54c3663e2864fe, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:37.406041 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca51300-3618-476b-89f3-80dca771f122 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:37.436329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:37.455156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268c4d8-8149-4375-850f-c39d26cbc889 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:37.473815 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 684/1365] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:37 2022] PUT /identity/v3/projects/740eca8d522c4a3ab6a691e7444f8f9d/users/7046bfe24636407f9a54c3663e2864fe/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:37.488906 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268c4d8-8149-4375-850f-c39d26cbc889 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:37.684759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04af901-6500-470c-95f4-e484b059939e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:37.685477 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04af901-6500-470c-95f4-e484b059939e admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:37.690736 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04af901-6500-470c-95f4-e484b059939e admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:37.691056 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04af901-6500-470c-95f4-e484b059939e admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:37.691774 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04af901-6500-470c-95f4-e484b059939e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:37.699697 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04af901-6500-470c-95f4-e484b059939e admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:37.736518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca51300-3618-476b-89f3-80dca771f122 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:37.739017 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 682/1366] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:37.762576 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca51300-3618-476b-89f3-80dca771f122 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:37.826453 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea2d9b-eaa4-4466-9c92-a3cba68566ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:37.829086 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea2d9b-eaa4-4466-9c92-a3cba68566ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:37.829509 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea2d9b-eaa4-4466-9c92-a3cba68566ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:37.829639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea2d9b-eaa4-4466-9c92-a3cba68566ed admin admin] PATH_INFO: `/v3/projects/740eca8d522c4a3ab6a691e7444f8f9d/users/7046bfe24636407f9a54c3663e2864fe/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:37.840912 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea2d9b-eaa4-4466-9c92-a3cba68566ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=740eca8d522c4a3ab6a691e7444f8f9d, user_id=7046bfe24636407f9a54c3663e2864fe, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:37.852007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea2d9b-eaa4-4466-9c92-a3cba68566ed admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:37.896901 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 685/1367] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:37 2022] PUT /identity/v3/projects/740eca8d522c4a3ab6a691e7444f8f9d/users/7046bfe24636407f9a54c3663e2864fe/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:37.903103 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f7ed1-5526-4cdf-986c-519ec6bb0d32 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:37.903604 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f7ed1-5526-4cdf-986c-519ec6bb0d32 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:37.903700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f7ed1-5526-4cdf-986c-519ec6bb0d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:37.999759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-6d8f7ed1-5526-4cdf-986c-519ec6bb0d32 None None] MFA Rules not processed for user `7046bfe24636407f9a54c3663e2864fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:38.001565 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-6d8f7ed1-5526-4cdf-986c-519ec6bb0d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:38.136951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:38.186068 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea2d9b-eaa4-4466-9c92-a3cba68566ed admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:38.186504 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 683/1368] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:37 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:38.187955 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5eea2d9b-eaa4-4466-9c92-a3cba68566ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:38.314699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea2d9b-eaa4-4466-9c92-a3cba68566ed admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:38.329665 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41345006-dd78-4f22-8971-a4f9f0ba4261 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:38.330708 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41345006-dd78-4f22-8971-a4f9f0ba4261 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:38.332371 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41345006-dd78-4f22-8971-a4f9f0ba4261 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:38.332902 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41345006-dd78-4f22-8971-a4f9f0ba4261 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:38.362517 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41345006-dd78-4f22-8971-a4f9f0ba4261 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:38.366848 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41345006-dd78-4f22-8971-a4f9f0ba4261 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:38.404060 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 686/1369] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:38 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:38.528583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f7ed1-5526-4cdf-986c-519ec6bb0d32 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:38.530190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-6d8f7ed1-5526-4cdf-986c-519ec6bb0d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:38.549842 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f7ed1-5526-4cdf-986c-519ec6bb0d32 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:38.558171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41345006-dd78-4f22-8971-a4f9f0ba4261 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:38.559845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-41345006-dd78-4f22-8971-a4f9f0ba4261 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:38.561483 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58cbade-a1e3-4775-951d-763887addbd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:38.562389 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cbade-a1e3-4775-951d-763887addbd1 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:38.562865 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cbade-a1e3-4775-951d-763887addbd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:38.563533 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cbade-a1e3-4775-951d-763887addbd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:38.697364 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41345006-dd78-4f22-8971-a4f9f0ba4261 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:38.703288 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd6140b-ce7e-4aca-9773-3fac864856cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:38.703980 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6140b-ce7e-4aca-9773-3fac864856cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:38.704263 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6140b-ce7e-4aca-9773-3fac864856cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:38.704539 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6140b-ce7e-4aca-9773-3fac864856cf admin admin] PATH_INFO: `/v3/projects/861546743d064fcf94b72c6eb5e3b233` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:38.706074 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd6140b-ce7e-4aca-9773-3fac864856cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=861546743d064fcf94b72c6eb5e3b233)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:38.708754 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd6140b-ce7e-4aca-9773-3fac864856cf admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:38.736622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58cbade-a1e3-4775-951d-763887addbd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:38.738154 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58cbade-a1e3-4775-951d-763887addbd1 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:38.759325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 684/1370] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:38.807447 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cbd6140b-ce7e-4aca-9773-3fac864856cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '861546743d064fcf94b72c6eb5e3b233', 'request_id': 'req-cbd6140b-ce7e-4aca-9773-3fac864856cf'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:38.816111 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cbd6140b-ce7e-4aca-9773-3fac864856cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '861546743d064fcf94b72c6eb5e3b233', 'request_id': 'req-cbd6140b-ce7e-4aca-9773-3fac864856cf'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:38.830724 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cbd6140b-ce7e-4aca-9773-3fac864856cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '861546743d064fcf94b72c6eb5e3b233', 'request_id': 'req-cbd6140b-ce7e-4aca-9773-3fac864856cf'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:38.847167 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cbd6140b-ce7e-4aca-9773-3fac864856cf admin admin] The token cache is being invalidate because project 861546743d064fcf94b72c6eb5e3b233 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:35:38.847644 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cbd6140b-ce7e-4aca-9773-3fac864856cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:38.848467 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cbd6140b-ce7e-4aca-9773-3fac864856cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:38.851095 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 687/1371] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:35:38 2022] DELETE /identity/v3/projects/861546743d064fcf94b72c6eb5e3b233 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:35:38.855257 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58cbade-a1e3-4775-951d-763887addbd1 service cinder] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:38.872965 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58cbade-a1e3-4775-951d-763887addbd1 service cinder] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:39.041106 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf21fd-46c5-4ec3-a001-4e4fb74d448e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:39.041787 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf21fd-46c5-4ec3-a001-4e4fb74d448e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:39.043789 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf21fd-46c5-4ec3-a001-4e4fb74d448e admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:39.045312 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf21fd-46c5-4ec3-a001-4e4fb74d448e admin admin] PATH_INFO: `/v3/users/8bc7c49859bb489ab4d7d94467207ddd` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:39.046754 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bf21fd-46c5-4ec3-a001-4e4fb74d448e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bc7c49859bb489ab4d7d94467207ddd)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:39.055393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bf21fd-46c5-4ec3-a001-4e4fb74d448e admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:39.191788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-a1bf21fd-46c5-4ec3-a001-4e4fb74d448e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bc7c49859bb489ab4d7d94467207ddd'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:39.195409 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-a1bf21fd-46c5-4ec3-a001-4e4fb74d448e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bc7c49859bb489ab4d7d94467207ddd'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:39.216699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-a1bf21fd-46c5-4ec3-a001-4e4fb74d448e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bc7c49859bb489ab4d7d94467207ddd'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:39.218563 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-a1bf21fd-46c5-4ec3-a001-4e4fb74d448e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bc7c49859bb489ab4d7d94467207ddd'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:39.236035 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-a1bf21fd-46c5-4ec3-a001-4e4fb74d448e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bc7c49859bb489ab4d7d94467207ddd'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:39.309611 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 685/1372] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:35:38 2022] DELETE /identity/v3/users/8bc7c49859bb489ab4d7d94467207ddd => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:39.571757 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd6140b-ce7e-4aca-9773-3fac864856cf admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:39.574197 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cbd6140b-ce7e-4aca-9773-3fac864856cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:39.609385 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd6140b-ce7e-4aca-9773-3fac864856cf admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:39.666637 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:39.782608 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe85eb-77a2-44fe-86a5-acb30e749835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:39.783282 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe85eb-77a2-44fe-86a5-acb30e749835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:39.783562 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe85eb-77a2-44fe-86a5-acb30e749835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:39.783879 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe85eb-77a2-44fe-86a5-acb30e749835 admin admin] PATH_INFO: `/v3/users/038007fbad164163bbf856424b59cd63` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:39.785996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe85eb-77a2-44fe-86a5-acb30e749835 admin admin] RBAC: Authorizing `identity:delete_user(user_id=038007fbad164163bbf856424b59cd63)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:39.790125 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe85eb-77a2-44fe-86a5-acb30e749835 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:39.919488 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-59fe85eb-77a2-44fe-86a5-acb30e749835 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '038007fbad164163bbf856424b59cd63'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:39.921860 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-59fe85eb-77a2-44fe-86a5-acb30e749835 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '038007fbad164163bbf856424b59cd63'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:39.973685 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-59fe85eb-77a2-44fe-86a5-acb30e749835 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '038007fbad164163bbf856424b59cd63'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:39.999111 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-59fe85eb-77a2-44fe-86a5-acb30e749835 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '038007fbad164163bbf856424b59cd63'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:39.999831 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-59fe85eb-77a2-44fe-86a5-acb30e749835 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '038007fbad164163bbf856424b59cd63'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:40.044744 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 688/1373] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:35:39 2022] DELETE /identity/v3/users/038007fbad164163bbf856424b59cd63 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:40.049451 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892bafe-1843-4b9d-bdbb-c8d951954e92 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:40.050325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892bafe-1843-4b9d-bdbb-c8d951954e92 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:40.050733 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892bafe-1843-4b9d-bdbb-c8d951954e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:40.087222 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-6892bafe-1843-4b9d-bdbb-c8d951954e92 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:40.091528 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-6892bafe-1843-4b9d-bdbb-c8d951954e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:40.102021 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:40.228489 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 686/1374] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:40.252432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe85eb-77a2-44fe-86a5-acb30e749835 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:40.254863 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-59fe85eb-77a2-44fe-86a5-acb30e749835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:40.358685 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe85eb-77a2-44fe-86a5-acb30e749835 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:40.369320 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858524ac-666f-4e9a-b328-802ab25d1dc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:40.371043 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858524ac-666f-4e9a-b328-802ab25d1dc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:40.371563 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858524ac-666f-4e9a-b328-802ab25d1dc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:40.371845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858524ac-666f-4e9a-b328-802ab25d1dc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:40.388905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858524ac-666f-4e9a-b328-802ab25d1dc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:40.390384 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858524ac-666f-4e9a-b328-802ab25d1dc9 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:40.410622 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 689/1375] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:40.674222 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce00af6-95b8-49be-9be9-70eaecec65b5 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:40.674586 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce00af6-95b8-49be-9be9-70eaecec65b5 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:40.674867 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce00af6-95b8-49be-9be9-70eaecec65b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:40.837435 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-1ce00af6-95b8-49be-9be9-70eaecec65b5 None None] MFA Rules not processed for user `144e9801ac1f470083d11647652ccdf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:40.844786 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-1ce00af6-95b8-49be-9be9-70eaecec65b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:41.102845 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 687/1376] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 432 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:41.119475 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858524ac-666f-4e9a-b328-802ab25d1dc9 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:41.123204 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-858524ac-666f-4e9a-b328-802ab25d1dc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:41.260289 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:41.301134 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858524ac-666f-4e9a-b328-802ab25d1dc9 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:41.309675 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da09cad-fba3-4e4e-a5d9-e90a7d11386d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:41.310654 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da09cad-fba3-4e4e-a5d9-e90a7d11386d service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:41.311158 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da09cad-fba3-4e4e-a5d9-e90a7d11386d service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:41.312620 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da09cad-fba3-4e4e-a5d9-e90a7d11386d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:41.326095 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da09cad-fba3-4e4e-a5d9-e90a7d11386d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:41.328382 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da09cad-fba3-4e4e-a5d9-e90a7d11386d service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:41.342081 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 690/1377] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:41.993353 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce00af6-95b8-49be-9be9-70eaecec65b5 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:41.994956 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-1ce00af6-95b8-49be-9be9-70eaecec65b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:42.026477 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce00af6-95b8-49be-9be9-70eaecec65b5 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:42.043460 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb64d3d-9d30-4480-b765-6c49cc82c3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:42.044129 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb64d3d-9d30-4480-b765-6c49cc82c3de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:42.044406 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb64d3d-9d30-4480-b765-6c49cc82c3de admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:42.045089 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb64d3d-9d30-4480-b765-6c49cc82c3de admin admin] PATH_INFO: `/v3/projects/1216452f661044568b65a1f5370a7a95` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:42.049402 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb64d3d-9d30-4480-b765-6c49cc82c3de admin admin] RBAC: Authorizing `identity:delete_project(project_id=1216452f661044568b65a1f5370a7a95)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:42.051491 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb64d3d-9d30-4480-b765-6c49cc82c3de admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:42.175924 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-9eb64d3d-9d30-4480-b765-6c49cc82c3de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1216452f661044568b65a1f5370a7a95', 'request_id': 'req-9eb64d3d-9d30-4480-b765-6c49cc82c3de'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:42.189052 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-9eb64d3d-9d30-4480-b765-6c49cc82c3de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1216452f661044568b65a1f5370a7a95', 'request_id': 'req-9eb64d3d-9d30-4480-b765-6c49cc82c3de'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:42.210902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-9eb64d3d-9d30-4480-b765-6c49cc82c3de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1216452f661044568b65a1f5370a7a95', 'request_id': 'req-9eb64d3d-9d30-4480-b765-6c49cc82c3de'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:42.237640 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-9eb64d3d-9d30-4480-b765-6c49cc82c3de admin admin] The token cache is being invalidate because project 1216452f661044568b65a1f5370a7a95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:35:42.239003 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-9eb64d3d-9d30-4480-b765-6c49cc82c3de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:42.240747 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-9eb64d3d-9d30-4480-b765-6c49cc82c3de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:42.247585 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 688/1378] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:35:41 2022] DELETE /identity/v3/projects/1216452f661044568b65a1f5370a7a95 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:42.250090 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b50b4b-5b63-4e25-8f83-bd921479ebbe None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:42.250567 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b50b4b-5b63-4e25-8f83-bd921479ebbe None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:42.251403 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b50b4b-5b63-4e25-8f83-bd921479ebbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:42.291199 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0965a21-abdb-4571-a4f6-c289e15e640c None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:42.291743 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0965a21-abdb-4571-a4f6-c289e15e640c None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:42.292222 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0965a21-abdb-4571-a4f6-c289e15e640c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:42.330532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-85b50b4b-5b63-4e25-8f83-bd921479ebbe None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:42.334749 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-85b50b4b-5b63-4e25-8f83-bd921479ebbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:42.352284 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-a0965a21-abdb-4571-a4f6-c289e15e640c None None] MFA Rules not processed for user `fe56d1d4278c43cd96a39f7f75f373b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:42.354135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-a0965a21-abdb-4571-a4f6-c289e15e640c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:42.379357 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:42.523892 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 689/1379] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:42.525342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0965a21-abdb-4571-a4f6-c289e15e640c None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:42.529679 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-a0965a21-abdb-4571-a4f6-c289e15e640c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:42.550355 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 691/1380] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:42.550355 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b50b4b-5b63-4e25-8f83-bd921479ebbe None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:42.550355 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-85b50b4b-5b63-4e25-8f83-bd921479ebbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:42.592256 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0965a21-abdb-4571-a4f6-c289e15e640c None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:42.610968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dbb56a-9834-4133-8c09-75b25cd0854b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:42.613472 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dbb56a-9834-4133-8c09-75b25cd0854b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:42.613812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dbb56a-9834-4133-8c09-75b25cd0854b admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:42.618855 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dbb56a-9834-4133-8c09-75b25cd0854b admin admin] PATH_INFO: `/v3/projects/045a50eaf3e54cf3b8463922b7a3ec3a` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:42.621983 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dbb56a-9834-4133-8c09-75b25cd0854b admin admin] RBAC: Authorizing `identity:delete_project(project_id=045a50eaf3e54cf3b8463922b7a3ec3a)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:42.628383 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dbb56a-9834-4133-8c09-75b25cd0854b admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:42.741280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-84dbb56a-9834-4133-8c09-75b25cd0854b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '045a50eaf3e54cf3b8463922b7a3ec3a', 'request_id': 'req-84dbb56a-9834-4133-8c09-75b25cd0854b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:42.747893 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b50b4b-5b63-4e25-8f83-bd921479ebbe None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:42.758207 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-84dbb56a-9834-4133-8c09-75b25cd0854b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '045a50eaf3e54cf3b8463922b7a3ec3a', 'request_id': 'req-84dbb56a-9834-4133-8c09-75b25cd0854b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:42.785122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-84dbb56a-9834-4133-8c09-75b25cd0854b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '045a50eaf3e54cf3b8463922b7a3ec3a', 'request_id': 'req-84dbb56a-9834-4133-8c09-75b25cd0854b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:42.821803 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-84dbb56a-9834-4133-8c09-75b25cd0854b admin admin] The token cache is being invalidate because project 045a50eaf3e54cf3b8463922b7a3ec3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:35:42.822106 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-84dbb56a-9834-4133-8c09-75b25cd0854b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:42.825302 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-84dbb56a-9834-4133-8c09-75b25cd0854b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:42.830487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 690/1381] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:35:42 2022] DELETE /identity/v3/projects/045a50eaf3e54cf3b8463922b7a3ec3a => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:42.841081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5519f4-2a9f-4a19-a2f2-bb358794cb04 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:42.841081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5519f4-2a9f-4a19-a2f2-bb358794cb04 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:42.841081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5519f4-2a9f-4a19-a2f2-bb358794cb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:42.869246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:42.893116 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a356285d-a663-4b26-8d5f-256d9de95e78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:42.893264 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-ad5519f4-2a9f-4a19-a2f2-bb358794cb04 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:42.893901 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a356285d-a663-4b26-8d5f-256d9de95e78 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:42.894248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a356285d-a663-4b26-8d5f-256d9de95e78 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:42.894511 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-ad5519f4-2a9f-4a19-a2f2-bb358794cb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:42.894636 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a356285d-a663-4b26-8d5f-256d9de95e78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:42.898187 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a356285d-a663-4b26-8d5f-256d9de95e78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:43.043927 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a356285d-a663-4b26-8d5f-256d9de95e78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:43.047301 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a356285d-a663-4b26-8d5f-256d9de95e78 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:43.061700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 691/1382] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:43.064994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567152b-2b4b-4b29-8cca-83c5679d6a61 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:43.065292 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567152b-2b4b-4b29-8cca-83c5679d6a61 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:43.065567 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567152b-2b4b-4b29-8cca-83c5679d6a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:43.078411 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 692/1383] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:43.081063 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a356285d-a663-4b26-8d5f-256d9de95e78 service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:43.108238 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a356285d-a663-4b26-8d5f-256d9de95e78 service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:43.120154 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efbb45-a0a8-4ee1-abde-cb136eeaaddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:43.120845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efbb45-a0a8-4ee1-abde-cb136eeaaddc admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:43.121154 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efbb45-a0a8-4ee1-abde-cb136eeaaddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:43.121435 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efbb45-a0a8-4ee1-abde-cb136eeaaddc admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:43.122152 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01efbb45-a0a8-4ee1-abde-cb136eeaaddc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:35:43.122468 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01efbb45-a0a8-4ee1-abde-cb136eeaaddc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:43.126827 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01efbb45-a0a8-4ee1-abde-cb136eeaaddc admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:43.132511 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-2567152b-2b4b-4b29-8cca-83c5679d6a61 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:43.134363 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-2567152b-2b4b-4b29-8cca-83c5679d6a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:43.158950 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 693/1384] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:35:43.159876 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efbb45-a0a8-4ee1-abde-cb136eeaaddc admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:43.160967 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 692/1385] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:43.165416 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ec4ca-507b-437b-ae08-98d3c5988275 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:43.165699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ec4ca-507b-437b-ae08-98d3c5988275 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:43.165973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ec4ca-507b-437b-ae08-98d3c5988275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:43.220753 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efbb45-a0a8-4ee1-abde-cb136eeaaddc admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:43.220753 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61890b0-f5dc-4f57-947d-61b15039b018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:43.220753 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61890b0-f5dc-4f57-947d-61b15039b018 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:43.220753 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61890b0-f5dc-4f57-947d-61b15039b018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:43.228334 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61890b0-f5dc-4f57-947d-61b15039b018 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:43.228334 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61890b0-f5dc-4f57-947d-61b15039b018 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:43.245167 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61890b0-f5dc-4f57-947d-61b15039b018 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:43.252044 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-a83ec4ca-507b-437b-ae08-98d3c5988275 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:43.253299 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-a83ec4ca-507b-437b-ae08-98d3c5988275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:43.294948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 693/1386] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:43.297417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ec4ca-507b-437b-ae08-98d3c5988275 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:43.325175 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 694/1387] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:35:43 2022] POST /identity/v3/projects => generated 407 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:43.326985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ec4ca-507b-437b-ae08-98d3c5988275 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:43.328789 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61890b0-f5dc-4f57-947d-61b15039b018 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:43.396121 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:43.503740 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8cd3db-f51c-470c-9b1f-a7a56bda70f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:43.509217 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8cd3db-f51c-470c-9b1f-a7a56bda70f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:43.509217 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8cd3db-f51c-470c-9b1f-a7a56bda70f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:43.509217 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8cd3db-f51c-470c-9b1f-a7a56bda70f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:43.509217 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8cd3db-f51c-470c-9b1f-a7a56bda70f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:35:43.509217 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8cd3db-f51c-470c-9b1f-a7a56bda70f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:43.516791 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8cd3db-f51c-470c-9b1f-a7a56bda70f6 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:43.557761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 694/1388] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:35:43.558836 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8cd3db-f51c-470c-9b1f-a7a56bda70f6 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:43.590165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8cd3db-f51c-470c-9b1f-a7a56bda70f6 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:43.598608 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61890b0-f5dc-4f57-947d-61b15039b018 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:43.613101 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa12dcf-146d-46af-b083-d326f9bbece5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:43.615111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa12dcf-146d-46af-b083-d326f9bbece5 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:43.619563 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934e3a0-236d-472b-a773-28e6f6c3ebd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:43.620536 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa12dcf-146d-46af-b083-d326f9bbece5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:43.620971 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa12dcf-146d-46af-b083-d326f9bbece5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:43.621768 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934e3a0-236d-472b-a773-28e6f6c3ebd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:43.623829 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934e3a0-236d-472b-a773-28e6f6c3ebd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:43.624716 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934e3a0-236d-472b-a773-28e6f6c3ebd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:43.626715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa12dcf-146d-46af-b083-d326f9bbece5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:43.628086 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934e3a0-236d-472b-a773-28e6f6c3ebd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:43.633480 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa12dcf-146d-46af-b083-d326f9bbece5 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:43.636266 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934e3a0-236d-472b-a773-28e6f6c3ebd1 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:43.727518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 695/1389] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:35:43 2022] POST /identity/v3/users => generated 304 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:43.731657 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934e3a0-236d-472b-a773-28e6f6c3ebd1 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:43.754549 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 695/1390] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:35:43 2022] POST /identity/v3/projects => generated 379 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:43.759826 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa12dcf-146d-46af-b083-d326f9bbece5 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:43.779701 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934e3a0-236d-472b-a773-28e6f6c3ebd1 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:44.001651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa12dcf-146d-46af-b083-d326f9bbece5 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:44.019171 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bb72c-7df0-46e7-8802-89e0b34191b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:44.019813 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bb72c-7df0-46e7-8802-89e0b34191b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:44.022332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bb72c-7df0-46e7-8802-89e0b34191b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:44.022654 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bb72c-7df0-46e7-8802-89e0b34191b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:44.024481 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693bb72c-7df0-46e7-8802-89e0b34191b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:44.028995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693bb72c-7df0-46e7-8802-89e0b34191b3 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:44.037130 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:44.047591 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6231e7ae-c133-41e4-9355-a61054025e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:44.056314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231e7ae-c133-41e4-9355-a61054025e35 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:44.057502 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231e7ae-c133-41e4-9355-a61054025e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:44.057794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231e7ae-c133-41e4-9355-a61054025e35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:44.059303 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6231e7ae-c133-41e4-9355-a61054025e35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:44.066067 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6231e7ae-c133-41e4-9355-a61054025e35 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:44.099411 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 696/1391] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:35:43 2022] POST /identity/v3/users => generated 290 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:35:44.102553 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bb72c-7df0-46e7-8802-89e0b34191b3 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:44.107463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 696/1392] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:44.114379 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6231e7ae-c133-41e4-9355-a61054025e35 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:44.128804 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bb72c-7df0-46e7-8802-89e0b34191b3 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:44.147451 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6231e7ae-c133-41e4-9355-a61054025e35 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:44.147769 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c869ae-7344-4b0e-92d7-103dce8fe0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:44.156335 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c869ae-7344-4b0e-92d7-103dce8fe0a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:44.158755 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c869ae-7344-4b0e-92d7-103dce8fe0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:44.159125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c869ae-7344-4b0e-92d7-103dce8fe0a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:44.159913 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c869ae-7344-4b0e-92d7-103dce8fe0a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:44.162487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a61d2-e8cb-4172-9ddb-f073ae8d3d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:44.163328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a61d2-e8cb-4172-9ddb-f073ae8d3d9c admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:44.163782 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a61d2-e8cb-4172-9ddb-f073ae8d3d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:44.164228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a61d2-e8cb-4172-9ddb-f073ae8d3d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:44.165111 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a61d2-e8cb-4172-9ddb-f073ae8d3d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:44.165471 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c869ae-7344-4b0e-92d7-103dce8fe0a5 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:44.167116 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a61d2-e8cb-4172-9ddb-f073ae8d3d9c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:44.198983 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 697/1393] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:44.202907 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a61d2-e8cb-4172-9ddb-f073ae8d3d9c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:44.207853 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 697/1394] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:44.214980 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c869ae-7344-4b0e-92d7-103dce8fe0a5 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:44.224485 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a61d2-e8cb-4172-9ddb-f073ae8d3d9c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:44.238715 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68366b63-95f3-4136-8e88-6279898ac216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:44.239855 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68366b63-95f3-4136-8e88-6279898ac216 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:44.240636 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68366b63-95f3-4136-8e88-6279898ac216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:44.241168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68366b63-95f3-4136-8e88-6279898ac216 admin admin] PATH_INFO: `/v3/projects/1bd12da0dfb54e2695ae39da479c48b9/users/7b472d928c1a464287f187d8d047554b/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:44.249753 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c869ae-7344-4b0e-92d7-103dce8fe0a5 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:44.270328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de32f8-5df0-4daa-80ff-e37d1921d235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:44.271209 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de32f8-5df0-4daa-80ff-e37d1921d235 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:44.271500 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de32f8-5df0-4daa-80ff-e37d1921d235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:44.271777 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de32f8-5df0-4daa-80ff-e37d1921d235 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:44.272557 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de32f8-5df0-4daa-80ff-e37d1921d235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:44.274222 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de32f8-5df0-4daa-80ff-e37d1921d235 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:44.295185 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68366b63-95f3-4136-8e88-6279898ac216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bd12da0dfb54e2695ae39da479c48b9, user_id=7b472d928c1a464287f187d8d047554b, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:44.306585 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68366b63-95f3-4136-8e88-6279898ac216 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:44.319895 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 698/1395] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:44.323644 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de32f8-5df0-4daa-80ff-e37d1921d235 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:44.334735 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 698/1396] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:44 2022] PUT /identity/v3/projects/1bd12da0dfb54e2695ae39da479c48b9/users/7b472d928c1a464287f187d8d047554b/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:35:44.335908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68366b63-95f3-4136-8e88-6279898ac216 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:44.340182 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-68366b63-95f3-4136-8e88-6279898ac216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:44.344177 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de32f8-5df0-4daa-80ff-e37d1921d235 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:44.479148 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd2b6d7-d171-4787-95ff-92e42f85232e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:44.480774 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2b6d7-d171-4787-95ff-92e42f85232e admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:44.481307 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2b6d7-d171-4787-95ff-92e42f85232e admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:44.481842 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2b6d7-d171-4787-95ff-92e42f85232e admin admin] PATH_INFO: `/v3/projects/60156f86f9ef4412b163a495a2976b36/users/40a2697be5f14b39b23178e218b5b25d/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:44.487974 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:44.521270 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd2b6d7-d171-4787-95ff-92e42f85232e admin admin] RBAC: Authorizing `identity:create_grant(project_id=60156f86f9ef4412b163a495a2976b36, user_id=40a2697be5f14b39b23178e218b5b25d, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:44.524882 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd2b6d7-d171-4787-95ff-92e42f85232e admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:44.562348 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 699/1397] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:44 2022] PUT /identity/v3/projects/60156f86f9ef4412b163a495a2976b36/users/40a2697be5f14b39b23178e218b5b25d/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:44.565374 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72faa286-8e73-44f6-8b01-7519237345db None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:44.565795 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72faa286-8e73-44f6-8b01-7519237345db None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:44.566323 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72faa286-8e73-44f6-8b01-7519237345db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:44.590353 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68366b63-95f3-4136-8e88-6279898ac216 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:44.676629 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-72faa286-8e73-44f6-8b01-7519237345db None None] MFA Rules not processed for user `7b472d928c1a464287f187d8d047554b`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:44.678159 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-72faa286-8e73-44f6-8b01-7519237345db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:44.741761 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e53624-ca83-4005-8a0c-9de08d37989f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:44.742526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e53624-ca83-4005-8a0c-9de08d37989f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:44.742710 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e53624-ca83-4005-8a0c-9de08d37989f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:44.742984 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e53624-ca83-4005-8a0c-9de08d37989f admin admin] PATH_INFO: `/v3/users/584fc05e6c614e56a2d20ba78969a454` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:44.749547 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e53624-ca83-4005-8a0c-9de08d37989f admin admin] RBAC: Authorizing `identity:delete_user(user_id=584fc05e6c614e56a2d20ba78969a454)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:44.751450 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e53624-ca83-4005-8a0c-9de08d37989f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:44.893239 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 700/1398] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:44 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:44.893239 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fdf67-6e42-4f08-9872-3e56d530c0ae None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:44.893239 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fdf67-6e42-4f08-9872-3e56d530c0ae None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:44.893239 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fdf67-6e42-4f08-9872-3e56d530c0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:44.897624 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-69e53624-ca83-4005-8a0c-9de08d37989f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '584fc05e6c614e56a2d20ba78969a454'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:44.898400 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-69e53624-ca83-4005-8a0c-9de08d37989f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '584fc05e6c614e56a2d20ba78969a454'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:44.920278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-69e53624-ca83-4005-8a0c-9de08d37989f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '584fc05e6c614e56a2d20ba78969a454'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:44.937197 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-69e53624-ca83-4005-8a0c-9de08d37989f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '584fc05e6c614e56a2d20ba78969a454'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:44.939045 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-69e53624-ca83-4005-8a0c-9de08d37989f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '584fc05e6c614e56a2d20ba78969a454'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:44.956244 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:44.968151 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-cf3fdf67-6e42-4f08-9872-3e56d530c0ae None None] MFA Rules not processed for user `40a2697be5f14b39b23178e218b5b25d`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:44.969449 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-cf3fdf67-6e42-4f08-9872-3e56d530c0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:44.972046 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 699/1399] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:35:44 2022] DELETE /identity/v3/users/584fc05e6c614e56a2d20ba78969a454 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:44.975393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e53624-ca83-4005-8a0c-9de08d37989f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:44.978691 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-69e53624-ca83-4005-8a0c-9de08d37989f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:45.060981 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e53624-ca83-4005-8a0c-9de08d37989f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:45.085644 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ca750-0eee-4eb4-9270-dd74c8c4b072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:45.086538 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ca750-0eee-4eb4-9270-dd74c8c4b072 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:45.086953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ca750-0eee-4eb4-9270-dd74c8c4b072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:45.087359 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ca750-0eee-4eb4-9270-dd74c8c4b072 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:45.088478 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ca750-0eee-4eb4-9270-dd74c8c4b072 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:45.091127 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ca750-0eee-4eb4-9270-dd74c8c4b072 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:45.133036 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 701/1400] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:45.136669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182b45c-7c10-4e62-beb7-c46bb95a87ba None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:45.137312 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182b45c-7c10-4e62-beb7-c46bb95a87ba None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:45.139182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182b45c-7c10-4e62-beb7-c46bb95a87ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:45.141944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 700/1401] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:35:44 2022] POST /identity/v3/projects => generated 409 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:45.144216 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ca750-0eee-4eb4-9270-dd74c8c4b072 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:45.148485 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-328ca750-0eee-4eb4-9270-dd74c8c4b072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:45.189158 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-e182b45c-7c10-4e62-beb7-c46bb95a87ba None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:45.190543 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-e182b45c-7c10-4e62-beb7-c46bb95a87ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:45.304296 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ca750-0eee-4eb4-9270-dd74c8c4b072 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:45.314819 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d2d47-a39b-44d4-8230-1e7f428e90ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:45.315621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d2d47-a39b-44d4-8230-1e7f428e90ef admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:45.316482 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d2d47-a39b-44d4-8230-1e7f428e90ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:45.317163 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d2d47-a39b-44d4-8230-1e7f428e90ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:45.317569 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d2d47-a39b-44d4-8230-1e7f428e90ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:45.319607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d2d47-a39b-44d4-8230-1e7f428e90ef admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:45.334245 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 702/1402] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:45.334245 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182b45c-7c10-4e62-beb7-c46bb95a87ba None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:45.353957 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:45.374747 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182b45c-7c10-4e62-beb7-c46bb95a87ba None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:45.379418 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 701/1403] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:35:45 2022] POST /identity/v3/projects => generated 379 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:45.381295 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d2d47-a39b-44d4-8230-1e7f428e90ef admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:45.388438 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-113d2d47-a39b-44d4-8230-1e7f428e90ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:45.390173 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c765ee3-5df9-4278-9b5c-2393a7cd8549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:45.390835 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c765ee3-5df9-4278-9b5c-2393a7cd8549 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:45.391081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c765ee3-5df9-4278-9b5c-2393a7cd8549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:45.391356 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c765ee3-5df9-4278-9b5c-2393a7cd8549 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:45.392219 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c765ee3-5df9-4278-9b5c-2393a7cd8549 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:45.394210 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c765ee3-5df9-4278-9b5c-2393a7cd8549 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:45.465423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d2d47-a39b-44d4-8230-1e7f428e90ef admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:45.471554 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d1e47-0f7b-41f6-942d-8a7fba386a6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:45.472350 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d1e47-0f7b-41f6-942d-8a7fba386a6a service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:45.472634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d1e47-0f7b-41f6-942d-8a7fba386a6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:45.473233 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d1e47-0f7b-41f6-942d-8a7fba386a6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:45.486467 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:45.500459 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 703/1404] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:35:45 2022] POST /identity/v3/users => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:45.500459 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c765ee3-5df9-4278-9b5c-2393a7cd8549 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:45.523635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c765ee3-5df9-4278-9b5c-2393a7cd8549 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:45.597743 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d1e47-0f7b-41f6-942d-8a7fba386a6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:45.599255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d1e47-0f7b-41f6-942d-8a7fba386a6a service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:45.614862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 702/1405] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:45.616079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d1e47-0f7b-41f6-942d-8a7fba386a6a service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:45.640424 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d1e47-0f7b-41f6-942d-8a7fba386a6a service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:45.643959 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a7a09-89e7-4ef0-a588-32ede73d80e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:45.644597 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a7a09-89e7-4ef0-a588-32ede73d80e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:45.644901 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a7a09-89e7-4ef0-a588-32ede73d80e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:45.645577 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a7a09-89e7-4ef0-a588-32ede73d80e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:45.646381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a7a09-89e7-4ef0-a588-32ede73d80e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:45.648677 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a7a09-89e7-4ef0-a588-32ede73d80e8 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:45.649387 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b2beb-65ba-4de1-acfb-3537cc69be00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:45.650267 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b2beb-65ba-4de1-acfb-3537cc69be00 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:45.650720 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b2beb-65ba-4de1-acfb-3537cc69be00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:45.651163 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b2beb-65ba-4de1-acfb-3537cc69be00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:45.652026 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6b2beb-65ba-4de1-acfb-3537cc69be00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:45.653871 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6b2beb-65ba-4de1-acfb-3537cc69be00 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:45.674170 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 703/1406] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:45.679637 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b2beb-65ba-4de1-acfb-3537cc69be00 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:45.695953 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 704/1407] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:35:45 2022] POST /identity/v3/users => generated 296 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:45.696127 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b2beb-65ba-4de1-acfb-3537cc69be00 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:45.697525 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a7a09-89e7-4ef0-a588-32ede73d80e8 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:45.700624 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87afff-00b5-4be2-84be-2d5fbd0a9d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:45.701806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87afff-00b5-4be2-84be-2d5fbd0a9d08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:45.702161 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87afff-00b5-4be2-84be-2d5fbd0a9d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:45.702506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87afff-00b5-4be2-84be-2d5fbd0a9d08 admin admin] PATH_INFO: `/v3/projects/bc4a0eb4129c49ac8f8d27c154b9f4d3/users/253e1780fb0049c3a41597d000598e8a/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:45.715998 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a7a09-89e7-4ef0-a588-32ede73d80e8 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:45.725108 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e87afff-00b5-4be2-84be-2d5fbd0a9d08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc4a0eb4129c49ac8f8d27c154b9f4d3, user_id=253e1780fb0049c3a41597d000598e8a, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:45.728752 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e87afff-00b5-4be2-84be-2d5fbd0a9d08 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:45.730748 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fed810-9490-4b6f-ad1b-6b4f17a6ba11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:45.731460 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fed810-9490-4b6f-ad1b-6b4f17a6ba11 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:45.731823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fed810-9490-4b6f-ad1b-6b4f17a6ba11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:45.732175 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fed810-9490-4b6f-ad1b-6b4f17a6ba11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:45.732959 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fed810-9490-4b6f-ad1b-6b4f17a6ba11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:45.735396 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fed810-9490-4b6f-ad1b-6b4f17a6ba11 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:45.758253 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 704/1408] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:45 2022] PUT /identity/v3/projects/bc4a0eb4129c49ac8f8d27c154b9f4d3/users/253e1780fb0049c3a41597d000598e8a/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:35:45.760337 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87afff-00b5-4be2-84be-2d5fbd0a9d08 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:45.771702 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 705/1409] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:35:45.774542 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fed810-9490-4b6f-ad1b-6b4f17a6ba11 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:45.801111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fed810-9490-4b6f-ad1b-6b4f17a6ba11 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:45.939262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87afff-00b5-4be2-84be-2d5fbd0a9d08 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:45.954459 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6358869-ed7d-4b2d-8e16-9a94ee8d14c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:45.955879 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6358869-ed7d-4b2d-8e16-9a94ee8d14c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:45.956703 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6358869-ed7d-4b2d-8e16-9a94ee8d14c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:45.957014 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6358869-ed7d-4b2d-8e16-9a94ee8d14c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:45.959912 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6358869-ed7d-4b2d-8e16-9a94ee8d14c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:45.966035 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6358869-ed7d-4b2d-8e16-9a94ee8d14c2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:45.967160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9633fc-15f1-45c0-96a2-53b06a68c0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:45.969078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9633fc-15f1-45c0-96a2-53b06a68c0d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:45.970807 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9633fc-15f1-45c0-96a2-53b06a68c0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:45.970990 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9633fc-15f1-45c0-96a2-53b06a68c0d2 admin admin] PATH_INFO: `/v3/projects/f0af62f46c2e494c97163fff306c7308/users/9fa72b5eee064ec991692974218161d5/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:45.999385 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 705/1410] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:46.005300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6358869-ed7d-4b2d-8e16-9a94ee8d14c2 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:46.016788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9633fc-15f1-45c0-96a2-53b06a68c0d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0af62f46c2e494c97163fff306c7308, user_id=9fa72b5eee064ec991692974218161d5, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:46.026343 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9633fc-15f1-45c0-96a2-53b06a68c0d2 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:46.031906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6358869-ed7d-4b2d-8e16-9a94ee8d14c2 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:46.043529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786ade3-2709-4bfb-92eb-ea21a8a2fee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:46.044399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786ade3-2709-4bfb-92eb-ea21a8a2fee0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:46.044835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786ade3-2709-4bfb-92eb-ea21a8a2fee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:46.045140 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2786ade3-2709-4bfb-92eb-ea21a8a2fee0 admin admin] PATH_INFO: `/v3/projects/bc4a0eb4129c49ac8f8d27c154b9f4d3/users/253e1780fb0049c3a41597d000598e8a/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:46.059751 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 706/1411] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:45 2022] PUT /identity/v3/projects/f0af62f46c2e494c97163fff306c7308/users/9fa72b5eee064ec991692974218161d5/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:46.059751 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6daee-b20d-402a-ad4a-e1d23e582fac None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:46.059751 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6daee-b20d-402a-ad4a-e1d23e582fac None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:46.060056 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2786ade3-2709-4bfb-92eb-ea21a8a2fee0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc4a0eb4129c49ac8f8d27c154b9f4d3, user_id=253e1780fb0049c3a41597d000598e8a, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:46.060160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6daee-b20d-402a-ad4a-e1d23e582fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:46.062323 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2786ade3-2709-4bfb-92eb-ea21a8a2fee0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:46.094388 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 706/1412] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:46 2022] PUT /identity/v3/projects/bc4a0eb4129c49ac8f8d27c154b9f4d3/users/253e1780fb0049c3a41597d000598e8a/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:46.094388 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786ade3-2709-4bfb-92eb-ea21a8a2fee0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:46.117241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-32b6daee-b20d-402a-ad4a-e1d23e582fac None None] MFA Rules not processed for user `55526ddbea014e6cbf7c079cb204059e`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:46.120532 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-32b6daee-b20d-402a-ad4a-e1d23e582fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:46.202567 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:46.232303 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:46.256675 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 707/1413] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:46.258392 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e219e1c-230b-427b-8201-3dc7dafe6f77 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:46.258894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e219e1c-230b-427b-8201-3dc7dafe6f77 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:46.259631 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e219e1c-230b-427b-8201-3dc7dafe6f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:46.295921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-6e219e1c-230b-427b-8201-3dc7dafe6f77 None None] MFA Rules not processed for user `253e1780fb0049c3a41597d000598e8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:46.297582 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-6e219e1c-230b-427b-8201-3dc7dafe6f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:46.305769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2786ade3-2709-4bfb-92eb-ea21a8a2fee0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:46.335838 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eda23f-e8fb-4a63-b733-d581ef56ddfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:46.336710 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eda23f-e8fb-4a63-b733-d581ef56ddfd admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:46.337203 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eda23f-e8fb-4a63-b733-d581ef56ddfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:46.341692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eda23f-e8fb-4a63-b733-d581ef56ddfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:46.344052 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eda23f-e8fb-4a63-b733-d581ef56ddfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:46.347623 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eda23f-e8fb-4a63-b733-d581ef56ddfd admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:46.388188 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 707/1414] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:46.394015 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eda23f-e8fb-4a63-b733-d581ef56ddfd admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:46.431007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b2eda23f-e8fb-4a63-b733-d581ef56ddfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:46.474957 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 708/1415] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:46 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:46.474957 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e219e1c-230b-427b-8201-3dc7dafe6f77 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:46.498184 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e219e1c-230b-427b-8201-3dc7dafe6f77 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:46.509925 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca32939-9054-4f9f-a436-dd7a6f24036e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:46.511428 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca32939-9054-4f9f-a436-dd7a6f24036e admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:46.511718 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca32939-9054-4f9f-a436-dd7a6f24036e admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:46.511991 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca32939-9054-4f9f-a436-dd7a6f24036e admin admin] PATH_INFO: `/v3/projects/f0af62f46c2e494c97163fff306c7308/users/9fa72b5eee064ec991692974218161d5/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:46.522664 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca32939-9054-4f9f-a436-dd7a6f24036e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0af62f46c2e494c97163fff306c7308, user_id=9fa72b5eee064ec991692974218161d5, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:46.537621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca32939-9054-4f9f-a436-dd7a6f24036e admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:46.566358 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 709/1416] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:46 2022] PUT /identity/v3/projects/f0af62f46c2e494c97163fff306c7308/users/9fa72b5eee064ec991692974218161d5/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:46.568015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca32939-9054-4f9f-a436-dd7a6f24036e admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:46.573172 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-eca32939-9054-4f9f-a436-dd7a6f24036e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:46.682993 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eda23f-e8fb-4a63-b733-d581ef56ddfd admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:46.712897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca32939-9054-4f9f-a436-dd7a6f24036e admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:46.720736 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4c97e-34d5-48d8-b9b8-43d1cec8b87c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:46.720736 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4c97e-34d5-48d8-b9b8-43d1cec8b87c service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:46.720736 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4c97e-34d5-48d8-b9b8-43d1cec8b87c service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:46.721064 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4c97e-34d5-48d8-b9b8-43d1cec8b87c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:46.790701 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbbe4b-47d0-40f3-9dc9-d475bf85ecab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:46.791639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbbe4b-47d0-40f3-9dc9-d475bf85ecab service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:46.791987 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbbe4b-47d0-40f3-9dc9-d475bf85ecab service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:46.792323 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbbe4b-47d0-40f3-9dc9-d475bf85ecab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:46.865127 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4c97e-34d5-48d8-b9b8-43d1cec8b87c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:46.867163 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4c97e-34d5-48d8-b9b8-43d1cec8b87c service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:46.873457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:46.897513 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 710/1417] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:46 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:46.901503 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e8ce3-579f-44d3-9c36-73c2e5668009 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:46.902473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e8ce3-579f-44d3-9c36-73c2e5668009 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:46.908397 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e8ce3-579f-44d3-9c36-73c2e5668009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:46.914352 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cbbe4b-47d0-40f3-9dc9-d475bf85ecab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:46.916116 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cbbe4b-47d0-40f3-9dc9-d475bf85ecab service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:46.939580 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 708/1418] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:46 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:46.995392 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-f97e8ce3-579f-44d3-9c36-73c2e5668009 None None] MFA Rules not processed for user `9fa72b5eee064ec991692974218161d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:46.997842 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-f97e8ce3-579f-44d3-9c36-73c2e5668009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:47.139457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 711/1419] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:47.151979 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbbe4b-47d0-40f3-9dc9-d475bf85ecab service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:47.166116 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbbe4b-47d0-40f3-9dc9-d475bf85ecab service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:47.179252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8724a0-0417-4795-8623-fd3e10dbf26e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:47.179956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8724a0-0417-4795-8623-fd3e10dbf26e service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:47.180234 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8724a0-0417-4795-8623-fd3e10dbf26e service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:47.181216 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8724a0-0417-4795-8623-fd3e10dbf26e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:47.212313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8724a0-0417-4795-8623-fd3e10dbf26e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:47.214908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8724a0-0417-4795-8623-fd3e10dbf26e service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:47.246699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 709/1420] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:47 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:47.249086 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e8ce3-579f-44d3-9c36-73c2e5668009 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:47.250501 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-f97e8ce3-579f-44d3-9c36-73c2e5668009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:47.265438 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e8ce3-579f-44d3-9c36-73c2e5668009 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:47.270055 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263b6c6-1241-4709-b663-2c665d96b081 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:47.271424 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263b6c6-1241-4709-b663-2c665d96b081 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:47.271424 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263b6c6-1241-4709-b663-2c665d96b081 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:47.271686 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263b6c6-1241-4709-b663-2c665d96b081 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:47.274737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b263b6c6-1241-4709-b663-2c665d96b081 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:47.304525 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654f8de-31c7-4d70-9653-73fa83d25a0b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:47.304783 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654f8de-31c7-4d70-9653-73fa83d25a0b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:47.305122 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654f8de-31c7-4d70-9653-73fa83d25a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:47.353002 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:47.368667 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e654f8de-31c7-4d70-9653-73fa83d25a0b None None] MFA Rules not processed for user `6085945ca3304bc99b4d9ca8a37d5319`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:47.370379 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e654f8de-31c7-4d70-9653-73fa83d25a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:47.427294 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b263b6c6-1241-4709-b663-2c665d96b081 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:47.431851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b263b6c6-1241-4709-b663-2c665d96b081 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:47.468675 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 710/1421] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:47.469537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654f8de-31c7-4d70-9653-73fa83d25a0b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:47.469650 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 712/1422] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:47 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:47.480399 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45af6a2-d896-4cf2-8b5f-604986ba02d3 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:47.481026 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45af6a2-d896-4cf2-8b5f-604986ba02d3 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:47.481026 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45af6a2-d896-4cf2-8b5f-604986ba02d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:47.514079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654f8de-31c7-4d70-9653-73fa83d25a0b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:47.547476 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b20091-7189-4dc5-ade4-f5c779ef363e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:47.548111 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b20091-7189-4dc5-ade4-f5c779ef363e service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:47.548391 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b20091-7189-4dc5-ade4-f5c779ef363e service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:47.548723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b20091-7189-4dc5-ade4-f5c779ef363e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:47.577588 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-c45af6a2-d896-4cf2-8b5f-604986ba02d3 None None] MFA Rules not processed for user `6085945ca3304bc99b4d9ca8a37d5319`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:47.579961 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-c45af6a2-d896-4cf2-8b5f-604986ba02d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:47.617189 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 713/1423] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:35:47.639299 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45af6a2-d896-4cf2-8b5f-604986ba02d3 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:47.641633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-c45af6a2-d896-4cf2-8b5f-604986ba02d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:47.685096 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45af6a2-d896-4cf2-8b5f-604986ba02d3 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:47.709261 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2da1b8-e38c-4bad-ab99-8a4913210871 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:47.710093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2da1b8-e38c-4bad-ab99-8a4913210871 service nova] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:47.710197 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2da1b8-e38c-4bad-ab99-8a4913210871 service nova] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:47.710450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2da1b8-e38c-4bad-ab99-8a4913210871 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:47.731658 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b20091-7189-4dc5-ade4-f5c779ef363e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:47.734051 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b20091-7189-4dc5-ade4-f5c779ef363e service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:47.745458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2da1b8-e38c-4bad-ab99-8a4913210871 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:47.750889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2da1b8-e38c-4bad-ab99-8a4913210871 service nova] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:47.774489 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 711/1424] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:47.775294 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b20091-7189-4dc5-ade4-f5c779ef363e service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:47.778964 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-01b20091-7189-4dc5-ade4-f5c779ef363e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:47.810118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 714/1425] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:35:47.982992 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2fde2-d748-4f4f-b3bc-a192dba5a8c9 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:47.985831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2fde2-d748-4f4f-b3bc-a192dba5a8c9 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:47.986735 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2fde2-d748-4f4f-b3bc-a192dba5a8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:48.005370 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b20091-7189-4dc5-ade4-f5c779ef363e service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:48.011101 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1b410-d701-4290-b806-4f9c6ae61ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:48.012261 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1b410-d701-4290-b806-4f9c6ae61ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:48.021533 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1b410-d701-4290-b806-4f9c6ae61ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:48.022173 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1b410-d701-4290-b806-4f9c6ae61ab0 admin admin] PATH_INFO: `/v3/projects/8bf9e8d2f2394d39ab75f3b0dd031d76` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:48.025876 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1b410-d701-4290-b806-4f9c6ae61ab0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bf9e8d2f2394d39ab75f3b0dd031d76)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:48.034137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1b410-d701-4290-b806-4f9c6ae61ab0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:48.058392 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-3ba2fde2-d748-4f4f-b3bc-a192dba5a8c9 None None] MFA Rules not processed for user `fe56d1d4278c43cd96a39f7f75f373b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:48.067085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-3ba2fde2-d748-4f4f-b3bc-a192dba5a8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:48.191609 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:48.215374 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c2d1b410-d701-4290-b806-4f9c6ae61ab0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bf9e8d2f2394d39ab75f3b0dd031d76', 'request_id': 'req-c2d1b410-d701-4290-b806-4f9c6ae61ab0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:48.248934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c2d1b410-d701-4290-b806-4f9c6ae61ab0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bf9e8d2f2394d39ab75f3b0dd031d76', 'request_id': 'req-c2d1b410-d701-4290-b806-4f9c6ae61ab0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:48.269589 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c2d1b410-d701-4290-b806-4f9c6ae61ab0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bf9e8d2f2394d39ab75f3b0dd031d76', 'request_id': 'req-c2d1b410-d701-4290-b806-4f9c6ae61ab0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:48.298906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c2d1b410-d701-4290-b806-4f9c6ae61ab0 admin admin] The token cache is being invalidate because project 8bf9e8d2f2394d39ab75f3b0dd031d76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:35:48.301526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c2d1b410-d701-4290-b806-4f9c6ae61ab0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:48.304491 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c2d1b410-d701-4290-b806-4f9c6ae61ab0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:48.306609 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 712/1426] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:35:47 2022] DELETE /identity/v3/projects/8bf9e8d2f2394d39ab75f3b0dd031d76 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:48.310797 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1b410-d701-4290-b806-4f9c6ae61ab0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:48.312795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c2d1b410-d701-4290-b806-4f9c6ae61ab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:48.323519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 715/1427] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:48.579406 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1b410-d701-4290-b806-4f9c6ae61ab0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:48.589382 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70996fb5-386d-420a-b9d5-20501e59fa26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:48.592010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70996fb5-386d-420a-b9d5-20501e59fa26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:48.592302 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70996fb5-386d-420a-b9d5-20501e59fa26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:48.592562 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70996fb5-386d-420a-b9d5-20501e59fa26 admin admin] PATH_INFO: `/v3/users/7046bfe24636407f9a54c3663e2864fe` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:48.595448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70996fb5-386d-420a-b9d5-20501e59fa26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7046bfe24636407f9a54c3663e2864fe)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:48.600000 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70996fb5-386d-420a-b9d5-20501e59fa26 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:48.709768 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-70996fb5-386d-420a-b9d5-20501e59fa26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7046bfe24636407f9a54c3663e2864fe'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:48.710513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-70996fb5-386d-420a-b9d5-20501e59fa26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7046bfe24636407f9a54c3663e2864fe'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:48.738281 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-70996fb5-386d-420a-b9d5-20501e59fa26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7046bfe24636407f9a54c3663e2864fe'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:48.753234 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-70996fb5-386d-420a-b9d5-20501e59fa26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7046bfe24636407f9a54c3663e2864fe'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:48.754190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-70996fb5-386d-420a-b9d5-20501e59fa26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7046bfe24636407f9a54c3663e2864fe'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:48.803594 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 713/1428] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:35:48 2022] DELETE /identity/v3/users/7046bfe24636407f9a54c3663e2864fe => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:49.110136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f52273-bb80-4876-9231-cbc4ebc69ab5 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:49.110672 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f52273-bb80-4876-9231-cbc4ebc69ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:49.110672 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f52273-bb80-4876-9231-cbc4ebc69ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:49.138112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295802ba-dd2b-4f7c-a363-532dc35f8f1b None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:49.139213 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295802ba-dd2b-4f7c-a363-532dc35f8f1b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:49.139727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295802ba-dd2b-4f7c-a363-532dc35f8f1b None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:49.141996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 714/1429] 10.222.0.23 () {54 vars in 912 bytes} [Thu Oct 20 13:35:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 20 13:35:49.147682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33ebd3-2f27-4353-ba8f-7b2172716783 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:49.148633 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33ebd3-2f27-4353-ba8f-7b2172716783 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:49.149191 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33ebd3-2f27-4353-ba8f-7b2172716783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:49.153548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-e0f52273-bb80-4876-9231-cbc4ebc69ab5 None None] MFA Rules not processed for user `144e9801ac1f470083d11647652ccdf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:49.157684 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-e0f52273-bb80-4876-9231-cbc4ebc69ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:49.182673 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:49.266705 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-de33ebd3-2f27-4353-ba8f-7b2172716783 None None] MFA Rules not processed for user `8a101adfc7a94bf1a4777350e9210a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:49.268342 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-de33ebd3-2f27-4353-ba8f-7b2172716783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:49.286888 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 716/1430] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:49.395891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:49.448366 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 715/1431] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 20 13:35:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:49.470633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f52273-bb80-4876-9231-cbc4ebc69ab5 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:49.472093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-e0f52273-bb80-4876-9231-cbc4ebc69ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:49.509696 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f52273-bb80-4876-9231-cbc4ebc69ab5 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:49.510616 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e66fa-5c9b-4e4f-a4c9-c2a3b53ab507 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:49.510908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e66fa-5c9b-4e4f-a4c9-c2a3b53ab507 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:49.511250 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e66fa-5c9b-4e4f-a4c9-c2a3b53ab507 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:49.515036 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 716/1432] 10.222.0.23 () {54 vars in 912 bytes} [Thu Oct 20 13:35:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:35:49.524539 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964b418-0f68-4fdd-b694-fbb5411437bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:49.524775 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964b418-0f68-4fdd-b694-fbb5411437bf service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:49.528790 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964b418-0f68-4fdd-b694-fbb5411437bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:49.530107 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964b418-0f68-4fdd-b694-fbb5411437bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:49.531381 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7202a9-39d2-4f52-b9d6-b15a859eb3b4 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:49.531671 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7202a9-39d2-4f52-b9d6-b15a859eb3b4 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:49.531950 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7202a9-39d2-4f52-b9d6-b15a859eb3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:49.565691 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964b418-0f68-4fdd-b694-fbb5411437bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:49.569932 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964b418-0f68-4fdd-b694-fbb5411437bf service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:49.573038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-cc7202a9-39d2-4f52-b9d6-b15a859eb3b4 None None] MFA Rules not processed for user `8a101adfc7a94bf1a4777350e9210a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:49.574255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cc7202a9-39d2-4f52-b9d6-b15a859eb3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:49.588550 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 717/1433] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 20 13:35:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:35:49.597614 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 717/1434] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:49.614688 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7202a9-39d2-4f52-b9d6-b15a859eb3b4 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:49.618634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cc7202a9-39d2-4f52-b9d6-b15a859eb3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:49.642940 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7202a9-39d2-4f52-b9d6-b15a859eb3b4 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:49.662038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6aec3-64d4-4276-b588-80f960d64586 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:49.662718 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6aec3-64d4-4276-b588-80f960d64586 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:49.663002 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6aec3-64d4-4276-b588-80f960d64586 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:49.664331 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6aec3-64d4-4276-b588-80f960d64586 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:49.698931 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6aec3-64d4-4276-b588-80f960d64586 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:49.702566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6aec3-64d4-4276-b588-80f960d64586 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:49.724172 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 718/1435] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:50.615985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964b418-0f68-4fdd-b694-fbb5411437bf service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:50.618107 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-8964b418-0f68-4fdd-b694-fbb5411437bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:50.850337 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964b418-0f68-4fdd-b694-fbb5411437bf service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:50.867833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d938b4-b5f6-4089-8305-0127ddab61ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:50.869629 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d938b4-b5f6-4089-8305-0127ddab61ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:50.871334 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d938b4-b5f6-4089-8305-0127ddab61ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:50.872292 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d938b4-b5f6-4089-8305-0127ddab61ff admin admin] PATH_INFO: `/v3/projects/740eca8d522c4a3ab6a691e7444f8f9d` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:50.876543 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d938b4-b5f6-4089-8305-0127ddab61ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=740eca8d522c4a3ab6a691e7444f8f9d)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:50.880409 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d938b4-b5f6-4089-8305-0127ddab61ff admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:50.905118 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6aec3-64d4-4276-b588-80f960d64586 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:50.907566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-56a6aec3-64d4-4276-b588-80f960d64586 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:51.066621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-38d938b4-b5f6-4089-8305-0127ddab61ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '740eca8d522c4a3ab6a691e7444f8f9d', 'request_id': 'req-38d938b4-b5f6-4089-8305-0127ddab61ff'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:51.091271 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-38d938b4-b5f6-4089-8305-0127ddab61ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '740eca8d522c4a3ab6a691e7444f8f9d', 'request_id': 'req-38d938b4-b5f6-4089-8305-0127ddab61ff'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:51.123716 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-38d938b4-b5f6-4089-8305-0127ddab61ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '740eca8d522c4a3ab6a691e7444f8f9d', 'request_id': 'req-38d938b4-b5f6-4089-8305-0127ddab61ff'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:51.160304 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6aec3-64d4-4276-b588-80f960d64586 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:51.167098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-38d938b4-b5f6-4089-8305-0127ddab61ff admin admin] The token cache is being invalidate because project 740eca8d522c4a3ab6a691e7444f8f9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:35:51.168335 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-38d938b4-b5f6-4089-8305-0127ddab61ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:51.170636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-38d938b4-b5f6-4089-8305-0127ddab61ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:35:51.178275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 718/1436] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:35:50 2022] DELETE /identity/v3/projects/740eca8d522c4a3ab6a691e7444f8f9d => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:51.185580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16eb62-1646-4483-bfec-17d5564dc911 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:51.185580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16eb62-1646-4483-bfec-17d5564dc911 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:51.185580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16eb62-1646-4483-bfec-17d5564dc911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:51.213670 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-4c16eb62-1646-4483-bfec-17d5564dc911 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:51.215999 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-4c16eb62-1646-4483-bfec-17d5564dc911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:51.263231 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:51.314346 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 719/1437] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:51.316091 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba4502-6814-40f9-83c9-bf86c02dfc7c None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:51.316492 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba4502-6814-40f9-83c9-bf86c02dfc7c None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:51.316918 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba4502-6814-40f9-83c9-bf86c02dfc7c None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:51.323150 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 720/1438] 10.222.0.23 () {54 vars in 961 bytes} [Thu Oct 20 13:35:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:35:51.325152 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68fa452-1df3-46c9-bd4d-87918eddb10f None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:51.325546 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68fa452-1df3-46c9-bd4d-87918eddb10f None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:51.325927 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68fa452-1df3-46c9-bd4d-87918eddb10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:51.354147 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb1481-8b43-40ed-b957-3466bcdac501 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2440594934b64621a01c441133778155', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:51.354794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb1481-8b43-40ed-b957-3466bcdac501 service placement] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:51.355075 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb1481-8b43-40ed-b957-3466bcdac501 service placement] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:51.355345 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb1481-8b43-40ed-b957-3466bcdac501 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:51.358280 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-98eb1481-8b43-40ed-b957-3466bcdac501 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:51.376644 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-a68fa452-1df3-46c9-bd4d-87918eddb10f None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:51.379584 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-a68fa452-1df3-46c9-bd4d-87918eddb10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:51.389635 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb1481-8b43-40ed-b957-3466bcdac501 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:51.393090 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb1481-8b43-40ed-b957-3466bcdac501 service placement] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:51.393647 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 721/1439] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:51.398596 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a0bd6-f2bb-4993-afca-3b9db342ba5f None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:51.399012 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a0bd6-f2bb-4993-afca-3b9db342ba5f None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:51.399387 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a0bd6-f2bb-4993-afca-3b9db342ba5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:51.426797 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 719/1440] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:51.430384 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9c3d4-8e7d-46db-911f-697c75ae3d59 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:51.430793 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9c3d4-8e7d-46db-911f-697c75ae3d59 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:51.431326 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9c3d4-8e7d-46db-911f-697c75ae3d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:51.456045 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-1a7a0bd6-f2bb-4993-afca-3b9db342ba5f None None] MFA Rules not processed for user `2440594934b64621a01c441133778155`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:51.458234 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-1a7a0bd6-f2bb-4993-afca-3b9db342ba5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:51.484423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-4ce9c3d4-8e7d-46db-911f-697c75ae3d59 None None] MFA Rules not processed for user `55526ddbea014e6cbf7c079cb204059e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:51.485789 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4ce9c3d4-8e7d-46db-911f-697c75ae3d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:51.496111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 722/1441] 10.222.0.23 () {58 vars in 1058 bytes} [Thu Oct 20 13:35:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:51.498476 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7a0bd6-f2bb-4993-afca-3b9db342ba5f None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:51.515635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7a0bd6-f2bb-4993-afca-3b9db342ba5f None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:51.539812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a5c2d-3d4b-43eb-bf60-cbcff59602d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:51.540710 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a5c2d-3d4b-43eb-bf60-cbcff59602d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:51.541053 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a5c2d-3d4b-43eb-bf60-cbcff59602d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:51.541336 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a5c2d-3d4b-43eb-bf60-cbcff59602d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:51.542006 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a5c2d-3d4b-43eb-bf60-cbcff59602d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:35:51.542318 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a5c2d-3d4b-43eb-bf60-cbcff59602d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:51.544200 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a5c2d-3d4b-43eb-bf60-cbcff59602d0 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:51.560262 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 723/1442] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:35:51.560262 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a5c2d-3d4b-43eb-bf60-cbcff59602d0 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:51.579171 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a5c2d-3d4b-43eb-bf60-cbcff59602d0 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:51.587964 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753d124-4481-4a21-b86a-fb737507edfc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2440594934b64621a01c441133778155', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:51.589035 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753d124-4481-4a21-b86a-fb737507edfc service placement] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:51.589317 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753d124-4481-4a21-b86a-fb737507edfc service placement] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:51.589594 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753d124-4481-4a21-b86a-fb737507edfc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:51.611027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 720/1443] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:51.612137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9c3d4-8e7d-46db-911f-697c75ae3d59 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:51.622093 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:51.628927 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9c3d4-8e7d-46db-911f-697c75ae3d59 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:51.634870 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148a62c-c16a-4f47-8b0b-e23551d98453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:51.635611 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148a62c-c16a-4f47-8b0b-e23551d98453 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:51.636005 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148a62c-c16a-4f47-8b0b-e23551d98453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:51.636389 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148a62c-c16a-4f47-8b0b-e23551d98453 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:51.637179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d148a62c-c16a-4f47-8b0b-e23551d98453 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:51.639252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d148a62c-c16a-4f47-8b0b-e23551d98453 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:51.647775 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753d124-4481-4a21-b86a-fb737507edfc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:51.654627 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753d124-4481-4a21-b86a-fb737507edfc service placement] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:51.679209 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 721/1444] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:35:51 2022] POST /identity/v3/projects => generated 387 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:51.684237 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148a62c-c16a-4f47-8b0b-e23551d98453 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:51.685511 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 724/1445] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:51.919339 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148a62c-c16a-4f47-8b0b-e23551d98453 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:51.966099 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41528de-a247-43e2-9e1a-66f35edf794b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:51.969419 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41528de-a247-43e2-9e1a-66f35edf794b admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:51.970414 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41528de-a247-43e2-9e1a-66f35edf794b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:51.971121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41528de-a247-43e2-9e1a-66f35edf794b admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:51.972108 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41528de-a247-43e2-9e1a-66f35edf794b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:51.980248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41528de-a247-43e2-9e1a-66f35edf794b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:52.070784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 722/1446] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:35:51 2022] POST /identity/v3/users => generated 294 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:52.074981 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753d124-4481-4a21-b86a-fb737507edfc service placement] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:52.098144 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753d124-4481-4a21-b86a-fb737507edfc service placement] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:52.119800 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5426486-8c44-473f-9c17-e44b1e423aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:52.120552 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5426486-8c44-473f-9c17-e44b1e423aaf admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:52.123818 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5426486-8c44-473f-9c17-e44b1e423aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:52.124485 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5426486-8c44-473f-9c17-e44b1e423aaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:52.133799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5426486-8c44-473f-9c17-e44b1e423aaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:52.134527 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5426486-8c44-473f-9c17-e44b1e423aaf admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:52.207037 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 725/1447] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:52.214110 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41528de-a247-43e2-9e1a-66f35edf794b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:52.256663 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41528de-a247-43e2-9e1a-66f35edf794b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:52.280419 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8728a6-7331-4cfb-81b9-1bf3cc906b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:52.281122 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8728a6-7331-4cfb-81b9-1bf3cc906b34 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:52.281409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8728a6-7331-4cfb-81b9-1bf3cc906b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:52.281685 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8728a6-7331-4cfb-81b9-1bf3cc906b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:52.282658 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8728a6-7331-4cfb-81b9-1bf3cc906b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:52.284638 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8728a6-7331-4cfb-81b9-1bf3cc906b34 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:52.344355 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5426486-8c44-473f-9c17-e44b1e423aaf admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:52.346642 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 723/1448] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:52.363500 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5426486-8c44-473f-9c17-e44b1e423aaf admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:52.381665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612ee4a-b0ef-438c-a542-012a07ff9d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:52.382438 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612ee4a-b0ef-438c-a542-012a07ff9d51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:52.382722 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612ee4a-b0ef-438c-a542-012a07ff9d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:52.382996 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612ee4a-b0ef-438c-a542-012a07ff9d51 admin admin] PATH_INFO: `/v3/projects/c10efd5e0b294676a2d248560ec75380/users/645881c853134cbf80305892f5c0b93d/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:52.477019 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612ee4a-b0ef-438c-a542-012a07ff9d51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c10efd5e0b294676a2d248560ec75380, user_id=645881c853134cbf80305892f5c0b93d, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:52.494604 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612ee4a-b0ef-438c-a542-012a07ff9d51 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:52.550761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 726/1449] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:52 2022] PUT /identity/v3/projects/c10efd5e0b294676a2d248560ec75380/users/645881c853134cbf80305892f5c0b93d/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:35:52.569044 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee666a-18a5-4c41-a7f6-db7c1899000b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:52.569437 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee666a-18a5-4c41-a7f6-db7c1899000b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:52.569719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ee666a-18a5-4c41-a7f6-db7c1899000b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:52.686571 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b5ee666a-18a5-4c41-a7f6-db7c1899000b None None] MFA Rules not processed for user `645881c853134cbf80305892f5c0b93d`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:52.687863 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b5ee666a-18a5-4c41-a7f6-db7c1899000b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:52.870672 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 724/1450] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:52.874843 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612ee4a-b0ef-438c-a542-012a07ff9d51 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:53.046045 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612ee4a-b0ef-438c-a542-012a07ff9d51 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:53.055359 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831eaf7-14a7-4766-a07e-687e8b520ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:53.056409 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831eaf7-14a7-4766-a07e-687e8b520ac7 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:53.056903 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831eaf7-14a7-4766-a07e-687e8b520ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:53.057398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831eaf7-14a7-4766-a07e-687e8b520ac7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:53.058596 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831eaf7-14a7-4766-a07e-687e8b520ac7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:53.060946 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831eaf7-14a7-4766-a07e-687e8b520ac7 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:53.108901 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 727/1451] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:35:52 2022] POST /identity/v3/projects => generated 387 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:35:53.112039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee666a-18a5-4c41-a7f6-db7c1899000b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:53.128168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ee666a-18a5-4c41-a7f6-db7c1899000b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:53.229253 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:53.270167 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce13e9f-e17e-4651-9d6e-a088da41a66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:53.272613 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce13e9f-e17e-4651-9d6e-a088da41a66a admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:53.272931 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce13e9f-e17e-4651-9d6e-a088da41a66a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:53.275039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce13e9f-e17e-4651-9d6e-a088da41a66a admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:53.277644 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce13e9f-e17e-4651-9d6e-a088da41a66a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:53.285697 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce13e9f-e17e-4651-9d6e-a088da41a66a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:53.346612 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 725/1452] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:35:53 2022] POST /identity/v3/users => generated 294 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:35:53.350565 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831eaf7-14a7-4766-a07e-687e8b520ac7 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:53.374961 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831eaf7-14a7-4766-a07e-687e8b520ac7 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:53.384648 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f118dc7-82bc-4a55-b11d-ca59d4af9e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:53.386217 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f118dc7-82bc-4a55-b11d-ca59d4af9e1b admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:53.386820 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f118dc7-82bc-4a55-b11d-ca59d4af9e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:53.387326 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f118dc7-82bc-4a55-b11d-ca59d4af9e1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:53.388252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f118dc7-82bc-4a55-b11d-ca59d4af9e1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:53.393555 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f118dc7-82bc-4a55-b11d-ca59d4af9e1b admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:53.415769 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 728/1453] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:53.420450 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce13e9f-e17e-4651-9d6e-a088da41a66a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:53.447074 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce13e9f-e17e-4651-9d6e-a088da41a66a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:53.456806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f99ac-9162-40a8-bd12-f918b123e479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:53.458503 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f99ac-9162-40a8-bd12-f918b123e479 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:53.458503 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f99ac-9162-40a8-bd12-f918b123e479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:53.458665 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f99ac-9162-40a8-bd12-f918b123e479 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:53.459304 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944f99ac-9162-40a8-bd12-f918b123e479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:53.461881 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944f99ac-9162-40a8-bd12-f918b123e479 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:53.492135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 726/1454] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:53.493250 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f118dc7-82bc-4a55-b11d-ca59d4af9e1b admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:53.518378 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f118dc7-82bc-4a55-b11d-ca59d4af9e1b admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:53.538399 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995dfc22-e24b-4249-a72d-e8b183345f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:53.539251 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995dfc22-e24b-4249-a72d-e8b183345f1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:53.539682 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995dfc22-e24b-4249-a72d-e8b183345f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:53.540083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995dfc22-e24b-4249-a72d-e8b183345f1c admin admin] PATH_INFO: `/v3/projects/31739e727c2044baaf3440393d952cd1/users/c2ea053b5b6642a49310b02be970f1ae/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:53.579822 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995dfc22-e24b-4249-a72d-e8b183345f1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=31739e727c2044baaf3440393d952cd1, user_id=c2ea053b5b6642a49310b02be970f1ae, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:53.583763 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995dfc22-e24b-4249-a72d-e8b183345f1c admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:53.620927 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 729/1455] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:53 2022] PUT /identity/v3/projects/31739e727c2044baaf3440393d952cd1/users/c2ea053b5b6642a49310b02be970f1ae/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:35:53.625410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806138d6-23e4-4df7-8a11-cb6c5bc14c7e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:53.625891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806138d6-23e4-4df7-8a11-cb6c5bc14c7e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:53.626307 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806138d6-23e4-4df7-8a11-cb6c5bc14c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:53.697844 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-806138d6-23e4-4df7-8a11-cb6c5bc14c7e None None] MFA Rules not processed for user `c2ea053b5b6642a49310b02be970f1ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:53.699407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-806138d6-23e4-4df7-8a11-cb6c5bc14c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:53.991510 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995dfc22-e24b-4249-a72d-e8b183345f1c admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:53.992960 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 727/1456] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:54.008428 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:54.139388 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995dfc22-e24b-4249-a72d-e8b183345f1c admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:54.169676 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c36b55-f566-43a2-af25-f486efe8ffd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:54.172714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c36b55-f566-43a2-af25-f486efe8ffd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:54.174423 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c36b55-f566-43a2-af25-f486efe8ffd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:54.174952 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c36b55-f566-43a2-af25-f486efe8ffd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:54.175805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c36b55-f566-43a2-af25-f486efe8ffd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:54.180298 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c36b55-f566-43a2-af25-f486efe8ffd5 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:54.251421 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806138d6-23e4-4df7-8a11-cb6c5bc14c7e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:54.258512 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 730/1457] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:35:53 2022] POST /identity/v3/projects => generated 385 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:35:54.278140 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806138d6-23e4-4df7-8a11-cb6c5bc14c7e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:54.416501 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8a5c5-8210-436e-b6fe-8bbc6ee4a308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:54.419054 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8a5c5-8210-436e-b6fe-8bbc6ee4a308 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:54.419492 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8a5c5-8210-436e-b6fe-8bbc6ee4a308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:54.420518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8a5c5-8210-436e-b6fe-8bbc6ee4a308 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:54.421697 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8a5c5-8210-436e-b6fe-8bbc6ee4a308 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:54.425206 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8a5c5-8210-436e-b6fe-8bbc6ee4a308 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:54.493687 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 728/1458] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:35:54 2022] POST /identity/v3/users => generated 299 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:35:54.509561 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c36b55-f566-43a2-af25-f486efe8ffd5 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:54.534835 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c36b55-f566-43a2-af25-f486efe8ffd5 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:54.559100 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327def14-87c9-4cee-89f5-3bf849cc8bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:54.560797 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327def14-87c9-4cee-89f5-3bf849cc8bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:54.562235 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327def14-87c9-4cee-89f5-3bf849cc8bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:54.563844 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327def14-87c9-4cee-89f5-3bf849cc8bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:54.565922 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327def14-87c9-4cee-89f5-3bf849cc8bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:54.574273 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327def14-87c9-4cee-89f5-3bf849cc8bd7 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:54.617431 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 731/1459] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:54.622159 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8a5c5-8210-436e-b6fe-8bbc6ee4a308 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:54.639278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8a5c5-8210-436e-b6fe-8bbc6ee4a308 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:54.647101 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8edc41-b358-4047-a611-cdef43cf992d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:54.648311 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8edc41-b358-4047-a611-cdef43cf992d admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:54.648999 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8edc41-b358-4047-a611-cdef43cf992d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:54.649734 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8edc41-b358-4047-a611-cdef43cf992d admin admin] PATH_INFO: `/v3/projects/468eb1fd1fc148bc9f432bb17381f546/users/2ffbb406b06f44d4a38c1790fff5219c/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:54.675976 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8edc41-b358-4047-a611-cdef43cf992d admin admin] RBAC: Authorizing `identity:create_grant(project_id=468eb1fd1fc148bc9f432bb17381f546, user_id=2ffbb406b06f44d4a38c1790fff5219c, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:54.681420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8edc41-b358-4047-a611-cdef43cf992d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:54.704164 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 729/1460] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:54 2022] PUT /identity/v3/projects/468eb1fd1fc148bc9f432bb17381f546/users/2ffbb406b06f44d4a38c1790fff5219c/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:35:54.708380 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327def14-87c9-4cee-89f5-3bf849cc8bd7 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:54.719672 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327def14-87c9-4cee-89f5-3bf849cc8bd7 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:54.801675 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25da8d-2aa3-45c8-80e6-644ff33a35d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:54.802352 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25da8d-2aa3-45c8-80e6-644ff33a35d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:54.802743 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25da8d-2aa3-45c8-80e6-644ff33a35d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:54.803035 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25da8d-2aa3-45c8-80e6-644ff33a35d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:54.803677 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25da8d-2aa3-45c8-80e6-644ff33a35d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:54.806861 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25da8d-2aa3-45c8-80e6-644ff33a35d3 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:54.819141 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 732/1461] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:35:54.823176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8edc41-b358-4047-a611-cdef43cf992d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:54.834468 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8edc41-b358-4047-a611-cdef43cf992d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:54.841526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9cba9-8dc1-449b-a4ac-6777f29ba8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:54.842581 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9cba9-8dc1-449b-a4ac-6777f29ba8dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:54.842942 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9cba9-8dc1-449b-a4ac-6777f29ba8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:54.843323 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9cba9-8dc1-449b-a4ac-6777f29ba8dd admin admin] PATH_INFO: `/v3/projects/468eb1fd1fc148bc9f432bb17381f546/users/2ffbb406b06f44d4a38c1790fff5219c/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:54.845657 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f9cba9-8dc1-449b-a4ac-6777f29ba8dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=468eb1fd1fc148bc9f432bb17381f546, user_id=2ffbb406b06f44d4a38c1790fff5219c, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:54.849348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f9cba9-8dc1-449b-a4ac-6777f29ba8dd admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:54.863672 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 730/1462] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:35:54 2022] PUT /identity/v3/projects/468eb1fd1fc148bc9f432bb17381f546/users/2ffbb406b06f44d4a38c1790fff5219c/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:35:54.875681 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dc91f-5be8-4f6a-af8c-b8ebbdd6e31d None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:54.876050 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dc91f-5be8-4f6a-af8c-b8ebbdd6e31d None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:54.876337 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dc91f-5be8-4f6a-af8c-b8ebbdd6e31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:54.911332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-f22dc91f-5be8-4f6a-af8c-b8ebbdd6e31d None None] MFA Rules not processed for user `2ffbb406b06f44d4a38c1790fff5219c`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:54.912564 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-f22dc91f-5be8-4f6a-af8c-b8ebbdd6e31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:55.007985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 733/1463] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:35:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:55.022724 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9cba9-8dc1-449b-a4ac-6777f29ba8dd admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:55.028054 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-22f9cba9-8dc1-449b-a4ac-6777f29ba8dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:55.045598 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:55.102607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9cba9-8dc1-449b-a4ac-6777f29ba8dd admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:55.106944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e744652-964c-40ed-a687-82c449976a9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:55.107490 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e744652-964c-40ed-a687-82c449976a9c service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:55.108466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e744652-964c-40ed-a687-82c449976a9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:55.108887 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e744652-964c-40ed-a687-82c449976a9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:55.122039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e744652-964c-40ed-a687-82c449976a9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:55.123896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e744652-964c-40ed-a687-82c449976a9c service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:55.138077 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 731/1464] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:55 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:35:55.900600 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dc91f-5be8-4f6a-af8c-b8ebbdd6e31d None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:55.906983 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:35:55.914174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dc91f-5be8-4f6a-af8c-b8ebbdd6e31d None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:55.918226 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd00272-b0b8-4ff2-a964-f37c3f59d193 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:55.918902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd00272-b0b8-4ff2-a964-f37c3f59d193 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:55.919551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd00272-b0b8-4ff2-a964-f37c3f59d193 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:55.920023 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd00272-b0b8-4ff2-a964-f37c3f59d193 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:55.990653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd00272-b0b8-4ff2-a964-f37c3f59d193 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:55.992131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd00272-b0b8-4ff2-a964-f37c3f59d193 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:56.008139 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 734/1465] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:35:55 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:35:58.237600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842e572-c2a6-4f21-9f8b-2e905b78116f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:58.237600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842e572-c2a6-4f21-9f8b-2e905b78116f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:58.237600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842e572-c2a6-4f21-9f8b-2e905b78116f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:58.405930 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d842e572-c2a6-4f21-9f8b-2e905b78116f None None] MFA Rules not processed for user `85bb137d0fb74031a49a5c629b32c32b`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:35:58.406374 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d842e572-c2a6-4f21-9f8b-2e905b78116f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:58.593342 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 732/1466] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 20 13:35:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:35:58.831702 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd00272-b0b8-4ff2-a964-f37c3f59d193 service cinder] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:35:58.836905 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-9cd00272-b0b8-4ff2-a964-f37c3f59d193 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:35:59.099824 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd00272-b0b8-4ff2-a964-f37c3f59d193 service cinder] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:35:59.122357 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ecd66-2677-4bc5-88dd-669451b0b51f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9834211488354651803000c7e8fab80b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:35:59.123038 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ecd66-2677-4bc5-88dd-669451b0b51f service swift] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:35:59.125284 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ecd66-2677-4bc5-88dd-669451b0b51f service swift] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:35:59.125681 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ecd66-2677-4bc5-88dd-669451b0b51f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:35:59.141921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ecd66-2677-4bc5-88dd-669451b0b51f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:35:59.145855 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ecd66-2677-4bc5-88dd-669451b0b51f service swift] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:35:59.167466 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 735/1467] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 20 13:35:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 13:36:00.152552 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842e572-c2a6-4f21-9f8b-2e905b78116f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:00.156101 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d842e572-c2a6-4f21-9f8b-2e905b78116f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:00.349773 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f8dbb-17b8-4f74-8d55-8ebf700ac76a None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:00.350322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f8dbb-17b8-4f74-8d55-8ebf700ac76a None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:00.350473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f8dbb-17b8-4f74-8d55-8ebf700ac76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:00.418342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-0b7f8dbb-17b8-4f74-8d55-8ebf700ac76a None None] MFA Rules not processed for user `8a101adfc7a94bf1a4777350e9210a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:00.423224 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-0b7f8dbb-17b8-4f74-8d55-8ebf700ac76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:00.433680 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842e572-c2a6-4f21-9f8b-2e905b78116f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:00.450054 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64074367-8ac5-4783-9fe2-a6c1419a79dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:00.450993 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64074367-8ac5-4783-9fe2-a6c1419a79dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:00.453007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64074367-8ac5-4783-9fe2-a6c1419a79dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:00.455426 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64074367-8ac5-4783-9fe2-a6c1419a79dd admin admin] PATH_INFO: `/v3/users/7b472d928c1a464287f187d8d047554b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:00.461355 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64074367-8ac5-4783-9fe2-a6c1419a79dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b472d928c1a464287f187d8d047554b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:00.463988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64074367-8ac5-4783-9fe2-a6c1419a79dd admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:00.594534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:00.635485 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 736/1468] 10.222.0.20 () {58 vars in 1015 bytes} [Thu Oct 20 13:36:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:00.655020 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-64074367-8ac5-4783-9fe2-a6c1419a79dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b472d928c1a464287f187d8d047554b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:00.656650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-64074367-8ac5-4783-9fe2-a6c1419a79dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b472d928c1a464287f187d8d047554b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:00.679854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-64074367-8ac5-4783-9fe2-a6c1419a79dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b472d928c1a464287f187d8d047554b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:00.705760 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-64074367-8ac5-4783-9fe2-a6c1419a79dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b472d928c1a464287f187d8d047554b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:00.708792 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-64074367-8ac5-4783-9fe2-a6c1419a79dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b472d928c1a464287f187d8d047554b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:00.770920 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 733/1469] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:00 2022] DELETE /identity/v3/users/7b472d928c1a464287f187d8d047554b => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:00.775730 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e23741-ffbf-49fb-9776-34e40ca14e92 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:00.776424 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e23741-ffbf-49fb-9776-34e40ca14e92 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:00.776964 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e23741-ffbf-49fb-9776-34e40ca14e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:00.818956 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-13e23741-ffbf-49fb-9776-34e40ca14e92 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:00.820681 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-13e23741-ffbf-49fb-9776-34e40ca14e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:00.854462 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:01.068226 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 737/1470] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:36:01.083730 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64074367-8ac5-4783-9fe2-a6c1419a79dd admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:01.085444 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-64074367-8ac5-4783-9fe2-a6c1419a79dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:01.128774 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e23741-ffbf-49fb-9776-34e40ca14e92 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:01.130875 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-13e23741-ffbf-49fb-9776-34e40ca14e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:01.260533 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64074367-8ac5-4783-9fe2-a6c1419a79dd admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:01.272402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87367aca-29b7-4543-b3d8-12ee86781086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:01.273261 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87367aca-29b7-4543-b3d8-12ee86781086 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:01.273650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87367aca-29b7-4543-b3d8-12ee86781086 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:01.274052 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87367aca-29b7-4543-b3d8-12ee86781086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:01.277694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-87367aca-29b7-4543-b3d8-12ee86781086 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:01.297942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e23741-ffbf-49fb-9776-34e40ca14e92 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:01.306884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87367aca-29b7-4543-b3d8-12ee86781086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:01.308568 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87367aca-29b7-4543-b3d8-12ee86781086 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:01.310709 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a621a7-2389-443c-9c5d-500799b485ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:01.312723 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a621a7-2389-443c-9c5d-500799b485ea service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:01.313066 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a621a7-2389-443c-9c5d-500799b485ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:01.313354 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a621a7-2389-443c-9c5d-500799b485ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:01.335784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 734/1471] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:01.342209 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a621a7-2389-443c-9c5d-500799b485ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:01.344790 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a621a7-2389-443c-9c5d-500799b485ea service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:01.362831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 738/1472] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:36:01.402938 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87367aca-29b7-4543-b3d8-12ee86781086 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:01.404501 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-87367aca-29b7-4543-b3d8-12ee86781086 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:01.438352 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87367aca-29b7-4543-b3d8-12ee86781086 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:01.444952 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263ee6d-bfa7-4a35-9e13-a4a55b37c1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:01.445752 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263ee6d-bfa7-4a35-9e13-a4a55b37c1de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:01.446133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263ee6d-bfa7-4a35-9e13-a4a55b37c1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:01.446648 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263ee6d-bfa7-4a35-9e13-a4a55b37c1de admin admin] PATH_INFO: `/v3/users/40a2697be5f14b39b23178e218b5b25d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:01.448073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263ee6d-bfa7-4a35-9e13-a4a55b37c1de admin admin] RBAC: Authorizing `identity:delete_user(user_id=40a2697be5f14b39b23178e218b5b25d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:01.450236 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263ee6d-bfa7-4a35-9e13-a4a55b37c1de admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:01.559679 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5263ee6d-bfa7-4a35-9e13-a4a55b37c1de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40a2697be5f14b39b23178e218b5b25d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:01.561354 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5263ee6d-bfa7-4a35-9e13-a4a55b37c1de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40a2697be5f14b39b23178e218b5b25d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:01.571345 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5263ee6d-bfa7-4a35-9e13-a4a55b37c1de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40a2697be5f14b39b23178e218b5b25d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:01.583878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5263ee6d-bfa7-4a35-9e13-a4a55b37c1de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40a2697be5f14b39b23178e218b5b25d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:01.585570 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5263ee6d-bfa7-4a35-9e13-a4a55b37c1de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40a2697be5f14b39b23178e218b5b25d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:01.612458 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 735/1473] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:01 2022] DELETE /identity/v3/users/40a2697be5f14b39b23178e218b5b25d => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:01.617221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f12c7-5af3-4ad9-a01e-685b600f3164 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:01.617511 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f12c7-5af3-4ad9-a01e-685b600f3164 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:01.617965 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f12c7-5af3-4ad9-a01e-685b600f3164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:01.661220 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-b43f12c7-5af3-4ad9-a01e-685b600f3164 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:01.662715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b43f12c7-5af3-4ad9-a01e-685b600f3164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:01.780892 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 739/1474] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:01.800030 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263ee6d-bfa7-4a35-9e13-a4a55b37c1de admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:01.801564 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5263ee6d-bfa7-4a35-9e13-a4a55b37c1de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:01.837080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:01.971471 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263ee6d-bfa7-4a35-9e13-a4a55b37c1de admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:01.980155 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76574fde-c999-46a3-b86d-6015c9d5ac48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:01.980826 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76574fde-c999-46a3-b86d-6015c9d5ac48 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:01.983508 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76574fde-c999-46a3-b86d-6015c9d5ac48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:01.983817 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76574fde-c999-46a3-b86d-6015c9d5ac48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:02.016671 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76574fde-c999-46a3-b86d-6015c9d5ac48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:02.018677 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76574fde-c999-46a3-b86d-6015c9d5ac48 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:02.044958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 736/1475] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:02.690957 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f12c7-5af3-4ad9-a01e-685b600f3164 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:02.693716 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b43f12c7-5af3-4ad9-a01e-685b600f3164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:02.815527 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f12c7-5af3-4ad9-a01e-685b600f3164 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:02.822807 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfb66f-4d8c-493a-942a-be04cbc7ec1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:02.824790 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfb66f-4d8c-493a-942a-be04cbc7ec1f service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:02.825549 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfb66f-4d8c-493a-942a-be04cbc7ec1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:02.826299 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfb66f-4d8c-493a-942a-be04cbc7ec1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:02.829804 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-91cfb66f-4d8c-493a-942a-be04cbc7ec1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:02.886306 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:02.966914 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cfb66f-4d8c-493a-942a-be04cbc7ec1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:02.968567 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cfb66f-4d8c-493a-942a-be04cbc7ec1f service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:02.987443 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 740/1476] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:02 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:03.850215 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76574fde-c999-46a3-b86d-6015c9d5ac48 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:03.851909 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-76574fde-c999-46a3-b86d-6015c9d5ac48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:03.882674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76574fde-c999-46a3-b86d-6015c9d5ac48 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:03.896385 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf88e78-f71e-4f96-bd09-5a80b00aa978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:03.897066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf88e78-f71e-4f96-bd09-5a80b00aa978 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:03.897345 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf88e78-f71e-4f96-bd09-5a80b00aa978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:03.897679 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf88e78-f71e-4f96-bd09-5a80b00aa978 admin admin] PATH_INFO: `/v3/projects/60156f86f9ef4412b163a495a2976b36` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:03.899036 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf88e78-f71e-4f96-bd09-5a80b00aa978 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60156f86f9ef4412b163a495a2976b36)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:03.902845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf88e78-f71e-4f96-bd09-5a80b00aa978 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:04.051627 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfb66f-4d8c-493a-942a-be04cbc7ec1f service cinder] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:04.053087 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-91cfb66f-4d8c-493a-942a-be04cbc7ec1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:04.091952 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfb66f-4d8c-493a-942a-be04cbc7ec1f service cinder] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:04.127818 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fcf88e78-f71e-4f96-bd09-5a80b00aa978 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60156f86f9ef4412b163a495a2976b36', 'request_id': 'req-fcf88e78-f71e-4f96-bd09-5a80b00aa978'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:04.163981 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fcf88e78-f71e-4f96-bd09-5a80b00aa978 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60156f86f9ef4412b163a495a2976b36', 'request_id': 'req-fcf88e78-f71e-4f96-bd09-5a80b00aa978'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:04.180251 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fcf88e78-f71e-4f96-bd09-5a80b00aa978 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60156f86f9ef4412b163a495a2976b36', 'request_id': 'req-fcf88e78-f71e-4f96-bd09-5a80b00aa978'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:04.220250 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fcf88e78-f71e-4f96-bd09-5a80b00aa978 admin admin] The token cache is being invalidate because project 60156f86f9ef4412b163a495a2976b36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:04.220544 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fcf88e78-f71e-4f96-bd09-5a80b00aa978 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:04.222105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fcf88e78-f71e-4f96-bd09-5a80b00aa978 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:04.225637 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 737/1477] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:03 2022] DELETE /identity/v3/projects/60156f86f9ef4412b163a495a2976b36 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:04.229990 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf88e78-f71e-4f96-bd09-5a80b00aa978 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:04.233212 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fcf88e78-f71e-4f96-bd09-5a80b00aa978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:04.274750 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48682301-8025-4329-91c9-687b5e3bd9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:04.275453 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48682301-8025-4329-91c9-687b5e3bd9e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:04.275720 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48682301-8025-4329-91c9-687b5e3bd9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:04.276008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48682301-8025-4329-91c9-687b5e3bd9e5 admin admin] PATH_INFO: `/v3/projects/1bd12da0dfb54e2695ae39da479c48b9` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:04.278310 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48682301-8025-4329-91c9-687b5e3bd9e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bd12da0dfb54e2695ae39da479c48b9)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:04.282250 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48682301-8025-4329-91c9-687b5e3bd9e5 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:04.404712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-48682301-8025-4329-91c9-687b5e3bd9e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bd12da0dfb54e2695ae39da479c48b9', 'request_id': 'req-48682301-8025-4329-91c9-687b5e3bd9e5'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:04.413153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-48682301-8025-4329-91c9-687b5e3bd9e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bd12da0dfb54e2695ae39da479c48b9', 'request_id': 'req-48682301-8025-4329-91c9-687b5e3bd9e5'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:04.440151 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-48682301-8025-4329-91c9-687b5e3bd9e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bd12da0dfb54e2695ae39da479c48b9', 'request_id': 'req-48682301-8025-4329-91c9-687b5e3bd9e5'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:04.453435 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-48682301-8025-4329-91c9-687b5e3bd9e5 admin admin] The token cache is being invalidate because project 1bd12da0dfb54e2695ae39da479c48b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:04.453875 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-48682301-8025-4329-91c9-687b5e3bd9e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:04.455205 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-48682301-8025-4329-91c9-687b5e3bd9e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:04.457220 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 741/1478] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:04 2022] DELETE /identity/v3/projects/1bd12da0dfb54e2695ae39da479c48b9 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:04.460829 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48682301-8025-4329-91c9-687b5e3bd9e5 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:04.463797 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-48682301-8025-4329-91c9-687b5e3bd9e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:04.559233 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf88e78-f71e-4f96-bd09-5a80b00aa978 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:04.663574 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:04.721633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48682301-8025-4329-91c9-687b5e3bd9e5 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:04.746809 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f148e-aba6-4680-bdc0-6b0e0a784ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:04.751037 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f148e-aba6-4680-bdc0-6b0e0a784ec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:04.751491 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f148e-aba6-4680-bdc0-6b0e0a784ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:04.751786 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f148e-aba6-4680-bdc0-6b0e0a784ec5 admin admin] PATH_INFO: `/v3/users/253e1780fb0049c3a41597d000598e8a` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:04.757118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f148e-aba6-4680-bdc0-6b0e0a784ec5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=253e1780fb0049c3a41597d000598e8a)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:04.760948 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:04.763814 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f148e-aba6-4680-bdc0-6b0e0a784ec5 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:04.806043 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1caf46-8665-497d-bedf-cc60d350e591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:04.806969 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1caf46-8665-497d-bedf-cc60d350e591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:04.808785 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1caf46-8665-497d-bedf-cc60d350e591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:04.817348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1caf46-8665-497d-bedf-cc60d350e591 admin admin] PATH_INFO: `/v3/users/9fa72b5eee064ec991692974218161d5` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:04.820556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1caf46-8665-497d-bedf-cc60d350e591 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fa72b5eee064ec991692974218161d5)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:04.830721 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1caf46-8665-497d-bedf-cc60d350e591 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:04.985493 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ec1caf46-8665-497d-bedf-cc60d350e591 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fa72b5eee064ec991692974218161d5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:04.986204 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ec1caf46-8665-497d-bedf-cc60d350e591 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fa72b5eee064ec991692974218161d5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:04.986551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-311f148e-aba6-4680-bdc0-6b0e0a784ec5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '253e1780fb0049c3a41597d000598e8a'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:04.987848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-311f148e-aba6-4680-bdc0-6b0e0a784ec5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '253e1780fb0049c3a41597d000598e8a'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:05.010561 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ec1caf46-8665-497d-bedf-cc60d350e591 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fa72b5eee064ec991692974218161d5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:05.012427 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-311f148e-aba6-4680-bdc0-6b0e0a784ec5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '253e1780fb0049c3a41597d000598e8a'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:05.013137 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-311f148e-aba6-4680-bdc0-6b0e0a784ec5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '253e1780fb0049c3a41597d000598e8a'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:05.026303 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-311f148e-aba6-4680-bdc0-6b0e0a784ec5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '253e1780fb0049c3a41597d000598e8a'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:05.029402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ec1caf46-8665-497d-bedf-cc60d350e591 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fa72b5eee064ec991692974218161d5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:05.032682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ec1caf46-8665-497d-bedf-cc60d350e591 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fa72b5eee064ec991692974218161d5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:05.082922 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 738/1479] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:04 2022] DELETE /identity/v3/users/9fa72b5eee064ec991692974218161d5 => generated 0 bytes in 854 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:05.089990 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 742/1480] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:04 2022] DELETE /identity/v3/users/253e1780fb0049c3a41597d000598e8a => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:05.585275 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbbe3f-f20a-4d23-8e91-9cf61ad94a31 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:05.585567 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbbe3f-f20a-4d23-8e91-9cf61ad94a31 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:05.585763 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbbe3f-f20a-4d23-8e91-9cf61ad94a31 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:05.587303 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 739/1481] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 20 13:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:36:05.673422 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c2ef6-94d6-48ab-bbc6-67538961a1c0 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:05.673905 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c2ef6-94d6-48ab-bbc6-67538961a1c0 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:05.674377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c2ef6-94d6-48ab-bbc6-67538961a1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:05.728812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-483c2ef6-94d6-48ab-bbc6-67538961a1c0 None None] MFA Rules not processed for user `ee5c2a3855cb4a668177aa19f337778d`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:05.730253 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-483c2ef6-94d6-48ab-bbc6-67538961a1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:05.824378 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbbe3f-f20a-4d23-8e91-9cf61ad94a31 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:05.827827 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a4cbbe3f-f20a-4d23-8e91-9cf61ad94a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:05.944464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 743/1482] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 20 13:36:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:05.955000 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbbe3f-f20a-4d23-8e91-9cf61ad94a31 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:05.962258 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3a44b-2ff0-4d40-87e8-c64f16c4e723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:05.963542 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3a44b-2ff0-4d40-87e8-c64f16c4e723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:05.964049 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3a44b-2ff0-4d40-87e8-c64f16c4e723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:05.964541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3a44b-2ff0-4d40-87e8-c64f16c4e723 admin admin] PATH_INFO: `/v3/users/645881c853134cbf80305892f5c0b93d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:05.966358 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a3a44b-2ff0-4d40-87e8-c64f16c4e723 admin admin] RBAC: Authorizing `identity:delete_user(user_id=645881c853134cbf80305892f5c0b93d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:05.969415 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a3a44b-2ff0-4d40-87e8-c64f16c4e723 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:06.086813 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-23a3a44b-2ff0-4d40-87e8-c64f16c4e723 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '645881c853134cbf80305892f5c0b93d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:06.087679 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-23a3a44b-2ff0-4d40-87e8-c64f16c4e723 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '645881c853134cbf80305892f5c0b93d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:06.112027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-23a3a44b-2ff0-4d40-87e8-c64f16c4e723 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '645881c853134cbf80305892f5c0b93d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:06.135142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-23a3a44b-2ff0-4d40-87e8-c64f16c4e723 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '645881c853134cbf80305892f5c0b93d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:06.135829 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-23a3a44b-2ff0-4d40-87e8-c64f16c4e723 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '645881c853134cbf80305892f5c0b93d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:06.151525 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:06.185594 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 740/1483] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:05 2022] DELETE /identity/v3/users/645881c853134cbf80305892f5c0b93d => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:36:06.190606 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59025504-0dd1-4f83-9a26-01fbec568904 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:06.191117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59025504-0dd1-4f83-9a26-01fbec568904 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:06.191439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59025504-0dd1-4f83-9a26-01fbec568904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:06.249870 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-59025504-0dd1-4f83-9a26-01fbec568904 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:06.251109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-59025504-0dd1-4f83-9a26-01fbec568904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:06.378345 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:06.401073 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 744/1484] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:06.474302 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3a44b-2ff0-4d40-87e8-c64f16c4e723 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:06.475945 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-23a3a44b-2ff0-4d40-87e8-c64f16c4e723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:06.546833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59025504-0dd1-4f83-9a26-01fbec568904 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:06.548538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-59025504-0dd1-4f83-9a26-01fbec568904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:06.633945 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3a44b-2ff0-4d40-87e8-c64f16c4e723 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:06.641553 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eff601-8313-417b-831f-32aed3ad4a12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:06.642772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eff601-8313-417b-831f-32aed3ad4a12 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:06.643205 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eff601-8313-417b-831f-32aed3ad4a12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:06.643774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eff601-8313-417b-831f-32aed3ad4a12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:06.659194 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eff601-8313-417b-831f-32aed3ad4a12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:06.661797 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eff601-8313-417b-831f-32aed3ad4a12 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:06.680699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 741/1485] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:06.803204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59025504-0dd1-4f83-9a26-01fbec568904 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:06.818620 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232b8f5-1441-421a-a425-9f8f5df9a489 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:06.820847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232b8f5-1441-421a-a425-9f8f5df9a489 service nova] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:06.821183 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232b8f5-1441-421a-a425-9f8f5df9a489 service nova] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:06.823524 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232b8f5-1441-421a-a425-9f8f5df9a489 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:06.826103 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-2232b8f5-1441-421a-a425-9f8f5df9a489 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:06.847369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232b8f5-1441-421a-a425-9f8f5df9a489 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:06.849601 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2232b8f5-1441-421a-a425-9f8f5df9a489 service nova] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:06.873024 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 745/1486] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:07.481017 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18140c66-6cd0-4875-83c1-8e975e76c1c0 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:07.481526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18140c66-6cd0-4875-83c1-8e975e76c1c0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:07.481673 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18140c66-6cd0-4875-83c1-8e975e76c1c0 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:07.483390 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 742/1487] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 20 13:36:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:36:07.520188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54abd5-b87e-412b-92f7-8fae4a240c09 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:07.521737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54abd5-b87e-412b-92f7-8fae4a240c09 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:07.522208 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54abd5-b87e-412b-92f7-8fae4a240c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:07.596703 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-de54abd5-b87e-412b-92f7-8fae4a240c09 None None] MFA Rules not processed for user `ee5c2a3855cb4a668177aa19f337778d`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:07.598540 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-de54abd5-b87e-412b-92f7-8fae4a240c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:07.626860 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 746/1488] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 20 13:36:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:07.673980 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18140c66-6cd0-4875-83c1-8e975e76c1c0 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:07.681442 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-18140c66-6cd0-4875-83c1-8e975e76c1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:07.740266 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18140c66-6cd0-4875-83c1-8e975e76c1c0 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:07.761375 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f7639-bdc8-4911-80be-e63355bddc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:07.763181 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f7639-bdc8-4911-80be-e63355bddc32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:07.763466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f7639-bdc8-4911-80be-e63355bddc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:07.763742 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f7639-bdc8-4911-80be-e63355bddc32 admin admin] PATH_INFO: `/v3/projects/f0af62f46c2e494c97163fff306c7308` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:07.768039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f7639-bdc8-4911-80be-e63355bddc32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0af62f46c2e494c97163fff306c7308)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:07.777291 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f7639-bdc8-4911-80be-e63355bddc32 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:07.925643 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3f7f7639-bdc8-4911-80be-e63355bddc32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0af62f46c2e494c97163fff306c7308', 'request_id': 'req-3f7f7639-bdc8-4911-80be-e63355bddc32'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:07.942052 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3f7f7639-bdc8-4911-80be-e63355bddc32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0af62f46c2e494c97163fff306c7308', 'request_id': 'req-3f7f7639-bdc8-4911-80be-e63355bddc32'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:07.955835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3f7f7639-bdc8-4911-80be-e63355bddc32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0af62f46c2e494c97163fff306c7308', 'request_id': 'req-3f7f7639-bdc8-4911-80be-e63355bddc32'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:07.978297 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3f7f7639-bdc8-4911-80be-e63355bddc32 admin admin] The token cache is being invalidate because project f0af62f46c2e494c97163fff306c7308 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:07.979277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3f7f7639-bdc8-4911-80be-e63355bddc32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:07.981277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3f7f7639-bdc8-4911-80be-e63355bddc32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:07.997539 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 743/1489] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:07 2022] DELETE /identity/v3/projects/f0af62f46c2e494c97163fff306c7308 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:08.002853 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3724c34-c143-4d2e-aa24-31e69582d62c None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:08.004377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3724c34-c143-4d2e-aa24-31e69582d62c None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:08.004804 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3724c34-c143-4d2e-aa24-31e69582d62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:08.088373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-e3724c34-c143-4d2e-aa24-31e69582d62c None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:08.092153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-e3724c34-c143-4d2e-aa24-31e69582d62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:08.225468 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f7639-bdc8-4911-80be-e63355bddc32 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:08.228330 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3f7f7639-bdc8-4911-80be-e63355bddc32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:08.266034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 747/1490] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:08.278671 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f5d71-39a2-47e0-97f7-e9509c74b3ba None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:08.279005 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f5d71-39a2-47e0-97f7-e9509c74b3ba None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:08.279284 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f5d71-39a2-47e0-97f7-e9509c74b3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:08.325733 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-586f5d71-39a2-47e0-97f7-e9509c74b3ba None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:08.327437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-586f5d71-39a2-47e0-97f7-e9509c74b3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:08.345338 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 748/1491] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:08.346854 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f5d71-39a2-47e0-97f7-e9509c74b3ba None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:08.407347 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f5d71-39a2-47e0-97f7-e9509c74b3ba None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:08.428942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed4246-40ce-4875-bf8e-fe0e7a419257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:08.429644 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed4246-40ce-4875-bf8e-fe0e7a419257 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:08.429933 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed4246-40ce-4875-bf8e-fe0e7a419257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:08.433286 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed4246-40ce-4875-bf8e-fe0e7a419257 admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:08.434768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed4246-40ce-4875-bf8e-fe0e7a419257 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:36:08.435088 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed4246-40ce-4875-bf8e-fe0e7a419257 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:08.444302 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed4246-40ce-4875-bf8e-fe0e7a419257 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:08.466941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 749/1492] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:36:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:36:08.468355 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed4246-40ce-4875-bf8e-fe0e7a419257 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:08.475099 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-e9ed4246-40ce-4875-bf8e-fe0e7a419257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:08.536282 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f7639-bdc8-4911-80be-e63355bddc32 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:08.548688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed4246-40ce-4875-bf8e-fe0e7a419257 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:08.551603 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40c39c-f223-4a58-9d03-0986027c60eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:08.552196 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40c39c-f223-4a58-9d03-0986027c60eb service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:08.552527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40c39c-f223-4a58-9d03-0986027c60eb service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:08.552734 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40c39c-f223-4a58-9d03-0986027c60eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:08.554694 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb38056-8f8c-4910-8206-aeed7ce84d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:08.555801 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb38056-8f8c-4910-8206-aeed7ce84d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:08.556105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb38056-8f8c-4910-8206-aeed7ce84d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:08.556389 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb38056-8f8c-4910-8206-aeed7ce84d35 admin admin] PATH_INFO: `/v3/projects/c10efd5e0b294676a2d248560ec75380` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:08.558591 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ee40c39c-f223-4a58-9d03-0986027c60eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:08.559011 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb38056-8f8c-4910-8206-aeed7ce84d35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c10efd5e0b294676a2d248560ec75380)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:08.561994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb38056-8f8c-4910-8206-aeed7ce84d35 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:08.591645 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40c39c-f223-4a58-9d03-0986027c60eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:08.593330 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40c39c-f223-4a58-9d03-0986027c60eb service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:08.600924 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:08.607652 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 744/1493] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:36:08.620324 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40c39c-f223-4a58-9d03-0986027c60eb service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:08.623596 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ee40c39c-f223-4a58-9d03-0986027c60eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:08.658743 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40c39c-f223-4a58-9d03-0986027c60eb service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:08.667252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-fdb38056-8f8c-4910-8206-aeed7ce84d35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c10efd5e0b294676a2d248560ec75380', 'request_id': 'req-fdb38056-8f8c-4910-8206-aeed7ce84d35'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:08.685830 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-fdb38056-8f8c-4910-8206-aeed7ce84d35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c10efd5e0b294676a2d248560ec75380', 'request_id': 'req-fdb38056-8f8c-4910-8206-aeed7ce84d35'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:08.688361 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:08.708532 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-fdb38056-8f8c-4910-8206-aeed7ce84d35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c10efd5e0b294676a2d248560ec75380', 'request_id': 'req-fdb38056-8f8c-4910-8206-aeed7ce84d35'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:08.725995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-fdb38056-8f8c-4910-8206-aeed7ce84d35 admin admin] The token cache is being invalidate because project c10efd5e0b294676a2d248560ec75380 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:08.727530 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-fdb38056-8f8c-4910-8206-aeed7ce84d35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:08.728318 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-fdb38056-8f8c-4910-8206-aeed7ce84d35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:08.736651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 750/1494] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:08 2022] DELETE /identity/v3/projects/c10efd5e0b294676a2d248560ec75380 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:08.738293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb38056-8f8c-4910-8206-aeed7ce84d35 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:08.744730 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-fdb38056-8f8c-4910-8206-aeed7ce84d35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:08.801304 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686af16d-54eb-4654-a293-ce63ba560785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:08.802155 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686af16d-54eb-4654-a293-ce63ba560785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:08.802588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686af16d-54eb-4654-a293-ce63ba560785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:08.803028 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686af16d-54eb-4654-a293-ce63ba560785 admin admin] PATH_INFO: `/v3/projects/bc4a0eb4129c49ac8f8d27c154b9f4d3` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:08.804378 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686af16d-54eb-4654-a293-ce63ba560785 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc4a0eb4129c49ac8f8d27c154b9f4d3)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:08.807258 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686af16d-54eb-4654-a293-ce63ba560785 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:08.869513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-686af16d-54eb-4654-a293-ce63ba560785 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc4a0eb4129c49ac8f8d27c154b9f4d3', 'request_id': 'req-686af16d-54eb-4654-a293-ce63ba560785'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:08.871901 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb38056-8f8c-4910-8206-aeed7ce84d35 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:08.876365 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-686af16d-54eb-4654-a293-ce63ba560785 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc4a0eb4129c49ac8f8d27c154b9f4d3', 'request_id': 'req-686af16d-54eb-4654-a293-ce63ba560785'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:08.888122 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-686af16d-54eb-4654-a293-ce63ba560785 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc4a0eb4129c49ac8f8d27c154b9f4d3', 'request_id': 'req-686af16d-54eb-4654-a293-ce63ba560785'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:08.907563 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-686af16d-54eb-4654-a293-ce63ba560785 admin admin] The token cache is being invalidate because project bc4a0eb4129c49ac8f8d27c154b9f4d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:08.908606 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-686af16d-54eb-4654-a293-ce63ba560785 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:08.910851 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-686af16d-54eb-4654-a293-ce63ba560785 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:08.912481 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 745/1495] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:08 2022] DELETE /identity/v3/projects/bc4a0eb4129c49ac8f8d27c154b9f4d3 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:08.913816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686af16d-54eb-4654-a293-ce63ba560785 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:08.916091 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-686af16d-54eb-4654-a293-ce63ba560785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:08.938480 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f8c92-cbb0-42cb-97af-ad8e431462d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:08.939265 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f8c92-cbb0-42cb-97af-ad8e431462d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:08.939682 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f8c92-cbb0-42cb-97af-ad8e431462d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:08.940089 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f8c92-cbb0-42cb-97af-ad8e431462d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:08.941069 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f8c92-cbb0-42cb-97af-ad8e431462d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:08.944011 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3f8c92-cbb0-42cb-97af-ad8e431462d9 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:08.977203 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 751/1496] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:08 2022] POST /identity/v3/projects => generated 363 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:08.978113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de82551-0da1-4377-a386-326e3622c36b None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:08.978404 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de82551-0da1-4377-a386-326e3622c36b None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:08.978686 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de82551-0da1-4377-a386-326e3622c36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:08.986794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686af16d-54eb-4654-a293-ce63ba560785 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:09.000509 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-3de82551-0da1-4377-a386-326e3622c36b None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:09.001654 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-3de82551-0da1-4377-a386-326e3622c36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:09.039253 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:09.067826 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ba762-17ee-4cec-bd1a-c24c78d4aec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:09.074644 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ba762-17ee-4cec-bd1a-c24c78d4aec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:09.074813 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 752/1497] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:09.075002 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ba762-17ee-4cec-bd1a-c24c78d4aec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:09.075002 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ba762-17ee-4cec-bd1a-c24c78d4aec7 admin admin] PATH_INFO: `/v3/users/c2ea053b5b6642a49310b02be970f1ae` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:09.075141 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de82551-0da1-4377-a386-326e3622c36b None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:09.075141 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-3de82551-0da1-4377-a386-326e3622c36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:09.077879 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ba762-17ee-4cec-bd1a-c24c78d4aec7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2ea053b5b6642a49310b02be970f1ae)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:09.080807 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ba762-17ee-4cec-bd1a-c24c78d4aec7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:09.086644 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de82551-0da1-4377-a386-326e3622c36b None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:09.093404 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb32014-3e60-4e64-93bb-66ba2f4df32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:09.094017 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb32014-3e60-4e64-93bb-66ba2f4df32d admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:09.094299 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb32014-3e60-4e64-93bb-66ba2f4df32d admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:09.094580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb32014-3e60-4e64-93bb-66ba2f4df32d admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:09.095318 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb32014-3e60-4e64-93bb-66ba2f4df32d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:09.099888 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb32014-3e60-4e64-93bb-66ba2f4df32d admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:09.123525 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:09.133197 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 753/1498] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:09 2022] POST /identity/v3/users => generated 282 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:09.134882 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74e877-fc38-43fb-957e-05dd0a2ba75c None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:09.135386 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74e877-fc38-43fb-957e-05dd0a2ba75c None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:09.136221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74e877-fc38-43fb-957e-05dd0a2ba75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:09.148476 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9c0ba762-17ee-4cec-bd1a-c24c78d4aec7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2ea053b5b6642a49310b02be970f1ae'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:09.149564 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9c0ba762-17ee-4cec-bd1a-c24c78d4aec7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2ea053b5b6642a49310b02be970f1ae'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:09.165924 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9c0ba762-17ee-4cec-bd1a-c24c78d4aec7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2ea053b5b6642a49310b02be970f1ae'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:09.166598 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-fb74e877-fc38-43fb-957e-05dd0a2ba75c None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:09.168082 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-fb74e877-fc38-43fb-957e-05dd0a2ba75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:09.174797 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9c0ba762-17ee-4cec-bd1a-c24c78d4aec7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2ea053b5b6642a49310b02be970f1ae'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:09.175669 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9c0ba762-17ee-4cec-bd1a-c24c78d4aec7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2ea053b5b6642a49310b02be970f1ae'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:09.177217 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 754/1499] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:09.177836 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74e877-fc38-43fb-957e-05dd0a2ba75c None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:09.179151 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-fb74e877-fc38-43fb-957e-05dd0a2ba75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:09.190960 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74e877-fc38-43fb-957e-05dd0a2ba75c None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:09.196146 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5a64d-619f-4ac8-8223-e374c7650ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:09.196762 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5a64d-619f-4ac8-8223-e374c7650ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:09.197075 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5a64d-619f-4ac8-8223-e374c7650ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:09.197347 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5a64d-619f-4ac8-8223-e374c7650ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:09.198147 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b5a64d-619f-4ac8-8223-e374c7650ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:09.199572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 746/1500] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:08 2022] DELETE /identity/v3/users/c2ea053b5b6642a49310b02be970f1ae => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:09.199787 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b5a64d-619f-4ac8-8223-e374c7650ed1 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:09.204421 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ba762-17ee-4cec-bd1a-c24c78d4aec7 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:09.214384 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 755/1501] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:09.218575 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5a64d-619f-4ac8-8223-e374c7650ed1 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:09.231411 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5a64d-619f-4ac8-8223-e374c7650ed1 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:09.296824 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ba762-17ee-4cec-bd1a-c24c78d4aec7 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:09.306930 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798e5ed-f879-476d-b451-ef1ae52dd709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:09.307377 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798e5ed-f879-476d-b451-ef1ae52dd709 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:09.307757 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798e5ed-f879-476d-b451-ef1ae52dd709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:09.308135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798e5ed-f879-476d-b451-ef1ae52dd709 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:09.308923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798e5ed-f879-476d-b451-ef1ae52dd709 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:36:09.309335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798e5ed-f879-476d-b451-ef1ae52dd709 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:09.311165 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798e5ed-f879-476d-b451-ef1ae52dd709 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:09.326413 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 747/1502] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:36:09.332529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798e5ed-f879-476d-b451-ef1ae52dd709 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:09.343638 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1381c-1694-47e3-bdec-55e2279d3e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:09.343915 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798e5ed-f879-476d-b451-ef1ae52dd709 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:09.344267 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1381c-1694-47e3-bdec-55e2279d3e4f admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:09.344550 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1381c-1694-47e3-bdec-55e2279d3e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:09.344899 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1381c-1694-47e3-bdec-55e2279d3e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:09.345533 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1381c-1694-47e3-bdec-55e2279d3e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:09.347151 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1381c-1694-47e3-bdec-55e2279d3e4f admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:09.349878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc91c1e-0f6d-4ab4-8b66-f02702c7a6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:09.349878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc91c1e-0f6d-4ab4-8b66-f02702c7a6bf admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:09.350919 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc91c1e-0f6d-4ab4-8b66-f02702c7a6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:09.351274 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc91c1e-0f6d-4ab4-8b66-f02702c7a6bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:09.352207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc91c1e-0f6d-4ab4-8b66-f02702c7a6bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:09.354483 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc91c1e-0f6d-4ab4-8b66-f02702c7a6bf admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:09.363101 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 756/1503] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:36:09.364882 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1381c-1694-47e3-bdec-55e2279d3e4f admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:09.376129 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1381c-1694-47e3-bdec-55e2279d3e4f admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:09.383728 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ce0a7-20fa-49d0-9f0a-c1ab59d83d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:09.384518 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ce0a7-20fa-49d0-9f0a-c1ab59d83d77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:09.384961 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ce0a7-20fa-49d0-9f0a-c1ab59d83d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:09.385341 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ce0a7-20fa-49d0-9f0a-c1ab59d83d77 admin admin] PATH_INFO: `/v3/projects/9c119266a2814a0d987a6215b980065f/users/bde7409927c14eacb88c209f1191c3e3/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:09.396391 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 748/1504] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:09 2022] POST /identity/v3/projects => generated 379 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:09.400772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc91c1e-0f6d-4ab4-8b66-f02702c7a6bf admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:09.404967 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ce0a7-20fa-49d0-9f0a-c1ab59d83d77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c119266a2814a0d987a6215b980065f, user_id=bde7409927c14eacb88c209f1191c3e3, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:09.407836 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ce0a7-20fa-49d0-9f0a-c1ab59d83d77 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:09.433114 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 757/1505] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:09 2022] PUT /identity/v3/projects/9c119266a2814a0d987a6215b980065f/users/bde7409927c14eacb88c209f1191c3e3/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:09.438087 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494b9b4-4dba-4a3c-bebe-198feb757d95 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:09.438403 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494b9b4-4dba-4a3c-bebe-198feb757d95 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:09.438702 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494b9b4-4dba-4a3c-bebe-198feb757d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:09.474183 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:09.495294 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-7494b9b4-4dba-4a3c-bebe-198feb757d95 None None] MFA Rules not processed for user `bde7409927c14eacb88c209f1191c3e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:09.497109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-7494b9b4-4dba-4a3c-bebe-198feb757d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:09.523800 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc91c1e-0f6d-4ab4-8b66-f02702c7a6bf admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:09.577686 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 758/1506] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:09.582531 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494b9b4-4dba-4a3c-bebe-198feb757d95 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:09.605503 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac610d7-d659-4e9f-b319-55afd16605e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:09.606472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac610d7-d659-4e9f-b319-55afd16605e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:09.606789 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac610d7-d659-4e9f-b319-55afd16605e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:09.607068 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac610d7-d659-4e9f-b319-55afd16605e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:09.607743 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac610d7-d659-4e9f-b319-55afd16605e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:09.609721 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac610d7-d659-4e9f-b319-55afd16605e1 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:09.636913 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 749/1507] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:09 2022] POST /identity/v3/users => generated 290 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:09.636913 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac610d7-d659-4e9f-b319-55afd16605e1 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:09.638637 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fac610d7-d659-4e9f-b319-55afd16605e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:09.639212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:09.650666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac610d7-d659-4e9f-b319-55afd16605e1 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:09.655233 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e896fe4-d24c-4dfd-9a28-bcdcd7c8f953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:09.655851 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e896fe4-d24c-4dfd-9a28-bcdcd7c8f953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:09.656128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e896fe4-d24c-4dfd-9a28-bcdcd7c8f953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:09.656400 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e896fe4-d24c-4dfd-9a28-bcdcd7c8f953 admin admin] PATH_INFO: `/v3/projects/31739e727c2044baaf3440393d952cd1` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:09.657612 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e896fe4-d24c-4dfd-9a28-bcdcd7c8f953 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31739e727c2044baaf3440393d952cd1)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:09.658566 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7494b9b4-4dba-4a3c-bebe-198feb757d95 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:09.659503 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e896fe4-d24c-4dfd-9a28-bcdcd7c8f953 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:09.664123 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd718375-9421-4462-8413-a4fd82cde237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:09.664729 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd718375-9421-4462-8413-a4fd82cde237 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:09.665046 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd718375-9421-4462-8413-a4fd82cde237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:09.665347 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd718375-9421-4462-8413-a4fd82cde237 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:09.666062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd718375-9421-4462-8413-a4fd82cde237 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:09.667989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd718375-9421-4462-8413-a4fd82cde237 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:09.699363 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 759/1508] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:09 2022] POST /identity/v3/projects => generated 363 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:36:09.699363 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd718375-9421-4462-8413-a4fd82cde237 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:09.708948 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6e896fe4-d24c-4dfd-9a28-bcdcd7c8f953 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31739e727c2044baaf3440393d952cd1', 'request_id': 'req-6e896fe4-d24c-4dfd-9a28-bcdcd7c8f953'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:09.715909 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6e896fe4-d24c-4dfd-9a28-bcdcd7c8f953 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31739e727c2044baaf3440393d952cd1', 'request_id': 'req-6e896fe4-d24c-4dfd-9a28-bcdcd7c8f953'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:09.724189 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6e896fe4-d24c-4dfd-9a28-bcdcd7c8f953 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31739e727c2044baaf3440393d952cd1', 'request_id': 'req-6e896fe4-d24c-4dfd-9a28-bcdcd7c8f953'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:09.735031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6e896fe4-d24c-4dfd-9a28-bcdcd7c8f953 admin admin] The token cache is being invalidate because project 31739e727c2044baaf3440393d952cd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:09.735321 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6e896fe4-d24c-4dfd-9a28-bcdcd7c8f953 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:09.735974 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6e896fe4-d24c-4dfd-9a28-bcdcd7c8f953 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:09.737532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 750/1509] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:09 2022] DELETE /identity/v3/projects/31739e727c2044baaf3440393d952cd1 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:09.738815 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e896fe4-d24c-4dfd-9a28-bcdcd7c8f953 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:09.740198 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6e896fe4-d24c-4dfd-9a28-bcdcd7c8f953 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:09.760258 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd718375-9421-4462-8413-a4fd82cde237 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:09.764827 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3d0d2-2c7b-479c-b242-c032d61fe434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:09.765476 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3d0d2-2c7b-479c-b242-c032d61fe434 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:09.765773 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3d0d2-2c7b-479c-b242-c032d61fe434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:09.766039 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3d0d2-2c7b-479c-b242-c032d61fe434 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:09.766687 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf3d0d2-2c7b-479c-b242-c032d61fe434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:09.768301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf3d0d2-2c7b-479c-b242-c032d61fe434 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:09.778354 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 760/1510] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:09.779172 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3d0d2-2c7b-479c-b242-c032d61fe434 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:09.780814 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-7bf3d0d2-2c7b-479c-b242-c032d61fe434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:09.794930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3d0d2-2c7b-479c-b242-c032d61fe434 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:09.795924 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:09.799936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f4b9d-f762-48ea-bdcc-9162ac32ae26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:09.800744 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f4b9d-f762-48ea-bdcc-9162ac32ae26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:09.801176 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f4b9d-f762-48ea-bdcc-9162ac32ae26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:09.801477 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f4b9d-f762-48ea-bdcc-9162ac32ae26 admin admin] PATH_INFO: `/v3/users/2ffbb406b06f44d4a38c1790fff5219c` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:09.802707 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f4b9d-f762-48ea-bdcc-9162ac32ae26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ffbb406b06f44d4a38c1790fff5219c)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:09.804674 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f4b9d-f762-48ea-bdcc-9162ac32ae26 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:09.812494 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e896fe4-d24c-4dfd-9a28-bcdcd7c8f953 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:09.817265 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71653f38-bbaf-4311-8c0e-79d1b83e6ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:09.817878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71653f38-bbaf-4311-8c0e-79d1b83e6ca3 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:09.818163 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71653f38-bbaf-4311-8c0e-79d1b83e6ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:09.818442 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71653f38-bbaf-4311-8c0e-79d1b83e6ca3 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:09.819079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71653f38-bbaf-4311-8c0e-79d1b83e6ca3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:09.821023 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71653f38-bbaf-4311-8c0e-79d1b83e6ca3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:09.846319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 751/1511] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:09 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:09.847170 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71653f38-bbaf-4311-8c0e-79d1b83e6ca3 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:09.851917 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-71653f38-bbaf-4311-8c0e-79d1b83e6ca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:09.856690 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-755f4b9d-f762-48ea-bdcc-9162ac32ae26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ffbb406b06f44d4a38c1790fff5219c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:09.857380 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-755f4b9d-f762-48ea-bdcc-9162ac32ae26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ffbb406b06f44d4a38c1790fff5219c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:09.866645 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-755f4b9d-f762-48ea-bdcc-9162ac32ae26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ffbb406b06f44d4a38c1790fff5219c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:09.867472 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-755f4b9d-f762-48ea-bdcc-9162ac32ae26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ffbb406b06f44d4a38c1790fff5219c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:09.867824 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71653f38-bbaf-4311-8c0e-79d1b83e6ca3 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:09.873000 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5cfab8-46ed-48bb-bfd2-9e339093f237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:09.873745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cfab8-46ed-48bb-bfd2-9e339093f237 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:09.874032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cfab8-46ed-48bb-bfd2-9e339093f237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:09.874312 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cfab8-46ed-48bb-bfd2-9e339093f237 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:09.875025 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5cfab8-46ed-48bb-bfd2-9e339093f237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:09.876656 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-755f4b9d-f762-48ea-bdcc-9162ac32ae26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ffbb406b06f44d4a38c1790fff5219c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:09.876764 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5cfab8-46ed-48bb-bfd2-9e339093f237 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:09.887610 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 752/1512] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:09.889876 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5cfab8-46ed-48bb-bfd2-9e339093f237 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:09.891717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ad5cfab8-46ed-48bb-bfd2-9e339093f237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:09.898937 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 761/1513] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:09 2022] DELETE /identity/v3/users/2ffbb406b06f44d4a38c1790fff5219c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:09.898937 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f4b9d-f762-48ea-bdcc-9162ac32ae26 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:09.898937 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-755f4b9d-f762-48ea-bdcc-9162ac32ae26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:09.906092 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5cfab8-46ed-48bb-bfd2-9e339093f237 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:09.950678 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:09.981783 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c5b22-22fb-4a6e-ac5d-a83a46603ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:09.982409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c5b22-22fb-4a6e-ac5d-a83a46603ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:09.982695 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c5b22-22fb-4a6e-ac5d-a83a46603ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:09.982969 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c5b22-22fb-4a6e-ac5d-a83a46603ec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:09.983712 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c5b22-22fb-4a6e-ac5d-a83a46603ec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:09.985507 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c5b22-22fb-4a6e-ac5d-a83a46603ec7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:09.990736 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f4b9d-f762-48ea-bdcc-9162ac32ae26 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:09.997688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b59efd-26ec-4d69-82f4-3aa9db476fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:09.998891 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b59efd-26ec-4d69-82f4-3aa9db476fc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:09.999177 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b59efd-26ec-4d69-82f4-3aa9db476fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:09.999451 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b59efd-26ec-4d69-82f4-3aa9db476fc4 admin admin] PATH_INFO: `/v3/projects/ec4cd4e97af548e68428ebadd9bfe51d/users/94799edd30b74d169c40e8e497804816/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:10.005156 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 753/1514] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:10.006692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c5b22-22fb-4a6e-ac5d-a83a46603ec7 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:10.018322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b59efd-26ec-4d69-82f4-3aa9db476fc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec4cd4e97af548e68428ebadd9bfe51d, user_id=94799edd30b74d169c40e8e497804816, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:10.022236 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c5b22-22fb-4a6e-ac5d-a83a46603ec7 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:10.022367 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b59efd-26ec-4d69-82f4-3aa9db476fc4 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:10.044973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 762/1515] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:09 2022] PUT /identity/v3/projects/ec4cd4e97af548e68428ebadd9bfe51d/users/94799edd30b74d169c40e8e497804816/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:10.048196 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e29dce-0ea4-44a4-83bf-c4a191a1c0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:10.048938 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e29dce-0ea4-44a4-83bf-c4a191a1c0e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:10.049332 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e29dce-0ea4-44a4-83bf-c4a191a1c0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:10.049624 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e29dce-0ea4-44a4-83bf-c4a191a1c0e0 admin admin] PATH_INFO: `/v3/projects/8adaf78b1f9f4d21a853a2cc14a26788/users/25d47307ee494cb3b0dba891fd194c50/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:10.050121 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab38b9-dc61-41e4-974d-752230dec0e5 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:10.050571 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab38b9-dc61-41e4-974d-752230dec0e5 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:10.051552 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab38b9-dc61-41e4-974d-752230dec0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:10.077134 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e29dce-0ea4-44a4-83bf-c4a191a1c0e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8adaf78b1f9f4d21a853a2cc14a26788, user_id=25d47307ee494cb3b0dba891fd194c50, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:10.079728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e29dce-0ea4-44a4-83bf-c4a191a1c0e0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:10.095735 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-0dab38b9-dc61-41e4-974d-752230dec0e5 None None] MFA Rules not processed for user `94799edd30b74d169c40e8e497804816`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:10.097224 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-0dab38b9-dc61-41e4-974d-752230dec0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:10.097830 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 754/1516] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:10 2022] PUT /identity/v3/projects/8adaf78b1f9f4d21a853a2cc14a26788/users/25d47307ee494cb3b0dba891fd194c50/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:10.102199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e29dce-0ea4-44a4-83bf-c4a191a1c0e0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:10.182343 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e29dce-0ea4-44a4-83bf-c4a191a1c0e0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:10.187372 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1be83c-dc3c-40c2-b931-7078888f2c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:10.187979 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1be83c-dc3c-40c2-b931-7078888f2c98 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:10.188389 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1be83c-dc3c-40c2-b931-7078888f2c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:10.188666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1be83c-dc3c-40c2-b931-7078888f2c98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:10.188998 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 763/1517] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:36:10.189348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1be83c-dc3c-40c2-b931-7078888f2c98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:10.191042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1be83c-dc3c-40c2-b931-7078888f2c98 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:10.192957 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab38b9-dc61-41e4-974d-752230dec0e5 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:10.205505 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 755/1518] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:36:10.206466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1be83c-dc3c-40c2-b931-7078888f2c98 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:10.206620 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab38b9-dc61-41e4-974d-752230dec0e5 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:10.212421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bda9ce-f92d-48f5-a048-6a4eb66c59d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:10.213616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bda9ce-f92d-48f5-a048-6a4eb66c59d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:10.214445 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bda9ce-f92d-48f5-a048-6a4eb66c59d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:10.214445 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bda9ce-f92d-48f5-a048-6a4eb66c59d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:10.215135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bda9ce-f92d-48f5-a048-6a4eb66c59d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:10.216548 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1be83c-dc3c-40c2-b931-7078888f2c98 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:10.217475 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bda9ce-f92d-48f5-a048-6a4eb66c59d5 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:10.221995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea715f3f-e7fa-49af-a0b2-54cb4eb1c758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:10.223640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea715f3f-e7fa-49af-a0b2-54cb4eb1c758 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:10.223798 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea715f3f-e7fa-49af-a0b2-54cb4eb1c758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:10.224111 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea715f3f-e7fa-49af-a0b2-54cb4eb1c758 admin admin] PATH_INFO: `/v3/projects/8adaf78b1f9f4d21a853a2cc14a26788/users/25d47307ee494cb3b0dba891fd194c50/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:10.226122 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea715f3f-e7fa-49af-a0b2-54cb4eb1c758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8adaf78b1f9f4d21a853a2cc14a26788, user_id=25d47307ee494cb3b0dba891fd194c50, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:10.228989 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea715f3f-e7fa-49af-a0b2-54cb4eb1c758 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:10.242091 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 756/1519] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:10 2022] PUT /identity/v3/projects/8adaf78b1f9f4d21a853a2cc14a26788/users/25d47307ee494cb3b0dba891fd194c50/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:10.255326 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 764/1520] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:10 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:10.255518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd145f2-36e2-4140-9367-81dc14199ad5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:10.255518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd145f2-36e2-4140-9367-81dc14199ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:10.255518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd145f2-36e2-4140-9367-81dc14199ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:10.257037 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bda9ce-f92d-48f5-a048-6a4eb66c59d5 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:10.293871 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8cd145f2-36e2-4140-9367-81dc14199ad5 None None] MFA Rules not processed for user `25d47307ee494cb3b0dba891fd194c50`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:10.295098 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8cd145f2-36e2-4140-9367-81dc14199ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:10.307140 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:10.342927 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bda9ce-f92d-48f5-a048-6a4eb66c59d5 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:10.348208 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87f2c7-0554-46a6-93ca-ac126fee9bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:10.348977 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87f2c7-0554-46a6-93ca-ac126fee9bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:10.349267 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87f2c7-0554-46a6-93ca-ac126fee9bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:10.349541 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87f2c7-0554-46a6-93ca-ac126fee9bd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:10.350228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87f2c7-0554-46a6-93ca-ac126fee9bd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:10.355272 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87f2c7-0554-46a6-93ca-ac126fee9bd0 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:10.382709 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 757/1521] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:10 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:10.383620 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd145f2-36e2-4140-9367-81dc14199ad5 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:10.388060 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8cd145f2-36e2-4140-9367-81dc14199ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:10.395919 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 765/1522] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:10 2022] POST /identity/v3/users => generated 290 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:10.397027 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87f2c7-0554-46a6-93ca-ac126fee9bd0 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:10.399318 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-8c87f2c7-0554-46a6-93ca-ac126fee9bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:10.403381 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd145f2-36e2-4140-9367-81dc14199ad5 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:10.409277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6509616-1766-4fef-8acd-48706a287149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:10.409911 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6509616-1766-4fef-8acd-48706a287149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:10.410521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6509616-1766-4fef-8acd-48706a287149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:10.410794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6509616-1766-4fef-8acd-48706a287149 admin admin] PATH_INFO: `/v3/projects/468eb1fd1fc148bc9f432bb17381f546` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:10.411928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6509616-1766-4fef-8acd-48706a287149 admin admin] RBAC: Authorizing `identity:delete_project(project_id=468eb1fd1fc148bc9f432bb17381f546)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:10.413940 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6509616-1766-4fef-8acd-48706a287149 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:10.423215 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:10.475128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a6509616-1766-4fef-8acd-48706a287149 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '468eb1fd1fc148bc9f432bb17381f546', 'request_id': 'req-a6509616-1766-4fef-8acd-48706a287149'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:10.477745 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87f2c7-0554-46a6-93ca-ac126fee9bd0 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:10.483219 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a6509616-1766-4fef-8acd-48706a287149 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '468eb1fd1fc148bc9f432bb17381f546', 'request_id': 'req-a6509616-1766-4fef-8acd-48706a287149'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:10.494407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a6509616-1766-4fef-8acd-48706a287149 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '468eb1fd1fc148bc9f432bb17381f546', 'request_id': 'req-a6509616-1766-4fef-8acd-48706a287149'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:10.520365 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a6509616-1766-4fef-8acd-48706a287149 admin admin] The token cache is being invalidate because project 468eb1fd1fc148bc9f432bb17381f546 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:10.520719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a6509616-1766-4fef-8acd-48706a287149 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:10.522104 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a6509616-1766-4fef-8acd-48706a287149 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:10.523720 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 758/1523] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:10 2022] DELETE /identity/v3/projects/468eb1fd1fc148bc9f432bb17381f546 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:10.530503 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6509616-1766-4fef-8acd-48706a287149 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:10.532415 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a6509616-1766-4fef-8acd-48706a287149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:10.581048 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f382e1d2-b78f-430a-965e-e3c1649855ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:10.581763 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f382e1d2-b78f-430a-965e-e3c1649855ce service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:10.582882 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f382e1d2-b78f-430a-965e-e3c1649855ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:10.583189 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f382e1d2-b78f-430a-965e-e3c1649855ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:10.585139 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-f382e1d2-b78f-430a-965e-e3c1649855ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:10.622985 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6509616-1766-4fef-8acd-48706a287149 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:10.629436 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca6e65-8036-43f1-9ae8-bfb907033428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:10.630283 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca6e65-8036-43f1-9ae8-bfb907033428 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:10.630712 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca6e65-8036-43f1-9ae8-bfb907033428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:10.631132 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca6e65-8036-43f1-9ae8-bfb907033428 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:10.631930 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca6e65-8036-43f1-9ae8-bfb907033428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:10.633740 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca6e65-8036-43f1-9ae8-bfb907033428 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:10.646358 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 759/1524] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:10.647945 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9857a57-33c3-4145-bd1b-2ce2ebdaa6cc None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:10.648437 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9857a57-33c3-4145-bd1b-2ce2ebdaa6cc None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:10.649087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9857a57-33c3-4145-bd1b-2ce2ebdaa6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:10.670468 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d9857a57-33c3-4145-bd1b-2ce2ebdaa6cc None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:10.671684 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d9857a57-33c3-4145-bd1b-2ce2ebdaa6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:10.677661 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f382e1d2-b78f-430a-965e-e3c1649855ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:10.679218 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f382e1d2-b78f-430a-965e-e3c1649855ce service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:10.689183 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 760/1525] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:10.689183 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9857a57-33c3-4145-bd1b-2ce2ebdaa6cc None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:10.692637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 766/1526] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:10 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:10.694144 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9857a57-33c3-4145-bd1b-2ce2ebdaa6cc None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:10.698942 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f8e4c-3c03-448f-830a-034b02c8e0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:10.699857 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f8e4c-3c03-448f-830a-034b02c8e0a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:10.700113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73e9d5-a6fa-4267-958c-33dfa87eef09 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:10.700232 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f8e4c-3c03-448f-830a-034b02c8e0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:10.700428 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73e9d5-a6fa-4267-958c-33dfa87eef09 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:10.700601 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f8e4c-3c03-448f-830a-034b02c8e0a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:10.700694 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73e9d5-a6fa-4267-958c-33dfa87eef09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:10.701402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f8e4c-3c03-448f-830a-034b02c8e0a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:10.703191 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f8e4c-3c03-448f-830a-034b02c8e0a4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:10.728891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 761/1527] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:10.728891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f8e4c-3c03-448f-830a-034b02c8e0a4 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:10.730946 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f8e4c-3c03-448f-830a-034b02c8e0a4 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:10.739562 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc78d8-5dc2-4e7f-ac13-0a80565297d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:10.740336 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-aa73e9d5-a6fa-4267-958c-33dfa87eef09 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:10.740447 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc78d8-5dc2-4e7f-ac13-0a80565297d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:10.740703 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc78d8-5dc2-4e7f-ac13-0a80565297d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:10.741144 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc78d8-5dc2-4e7f-ac13-0a80565297d5 admin admin] PATH_INFO: `/v3/projects/ecb5d0913cc14a0c895f8b826bfdea4f/users/60108e21df4c4a128c12bbeb47a85c7d/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:10.742001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-aa73e9d5-a6fa-4267-958c-33dfa87eef09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:10.746195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:10.752561 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 767/1528] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:10.757503 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73e9d5-a6fa-4267-958c-33dfa87eef09 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:10.763001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc78d8-5dc2-4e7f-ac13-0a80565297d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecb5d0913cc14a0c895f8b826bfdea4f, user_id=60108e21df4c4a128c12bbeb47a85c7d, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:10.766365 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc78d8-5dc2-4e7f-ac13-0a80565297d5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:10.769073 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73e9d5-a6fa-4267-958c-33dfa87eef09 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:10.777890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7f6eb-4855-419c-a335-35bee567477a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:10.778517 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7f6eb-4855-419c-a335-35bee567477a admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:10.778801 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7f6eb-4855-419c-a335-35bee567477a admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:10.779076 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7f6eb-4855-419c-a335-35bee567477a admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:10.779737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7f6eb-4855-419c-a335-35bee567477a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:36:10.780044 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7f6eb-4855-419c-a335-35bee567477a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:10.781864 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7f6eb-4855-419c-a335-35bee567477a admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:10.785518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 762/1529] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:10 2022] PUT /identity/v3/projects/ecb5d0913cc14a0c895f8b826bfdea4f/users/60108e21df4c4a128c12bbeb47a85c7d/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:10.797209 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cfe5b0-ae6a-4576-88c1-4e1950d6704f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:10.797688 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cfe5b0-ae6a-4576-88c1-4e1950d6704f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:10.798163 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cfe5b0-ae6a-4576-88c1-4e1950d6704f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:10.805178 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 768/1530] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:36:10.808322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7f6eb-4855-419c-a335-35bee567477a admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:10.810052 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-53e7f6eb-4855-419c-a335-35bee567477a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:10.823877 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7f6eb-4855-419c-a335-35bee567477a admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:10.827624 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a4cfe5b0-ae6a-4576-88c1-4e1950d6704f None None] MFA Rules not processed for user `60108e21df4c4a128c12bbeb47a85c7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:10.828879 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a4cfe5b0-ae6a-4576-88c1-4e1950d6704f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:10.873418 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95fdbe-d918-4c33-8db4-606996b676b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:10.874025 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95fdbe-d918-4c33-8db4-606996b676b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:10.874304 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95fdbe-d918-4c33-8db4-606996b676b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:10.874580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95fdbe-d918-4c33-8db4-606996b676b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:10.876321 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-4c95fdbe-d918-4c33-8db4-606996b676b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:10.893801 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:10.894688 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 763/1531] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:36:10.895679 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cfe5b0-ae6a-4576-88c1-4e1950d6704f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:10.939869 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c95fdbe-d918-4c33-8db4-606996b676b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:10.942001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c95fdbe-d918-4c33-8db4-606996b676b6 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:10.950940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 769/1532] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:36:10.951707 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95fdbe-d918-4c33-8db4-606996b676b6 service cinder] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:10.958191 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cfe5b0-ae6a-4576-88c1-4e1950d6704f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:10.962414 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95fdbe-d918-4c33-8db4-606996b676b6 service cinder] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:10.962545 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e07e0-2362-4d1c-be02-0e2c61671033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:10.963264 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e07e0-2362-4d1c-be02-0e2c61671033 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:10.963642 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e07e0-2362-4d1c-be02-0e2c61671033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:10.963924 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e07e0-2362-4d1c-be02-0e2c61671033 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:10.964554 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e07e0-2362-4d1c-be02-0e2c61671033 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:10.966479 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e07e0-2362-4d1c-be02-0e2c61671033 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:10.967761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb461d-2f73-4e76-975a-e762c6a0c398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:10.968361 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb461d-2f73-4e76-975a-e762c6a0c398 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:10.968643 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb461d-2f73-4e76-975a-e762c6a0c398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:10.968948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb461d-2f73-4e76-975a-e762c6a0c398 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:10.969571 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb461d-2f73-4e76-975a-e762c6a0c398 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:10.971491 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb461d-2f73-4e76-975a-e762c6a0c398 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:10.996799 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 764/1533] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:10 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:11.000776 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e07e0-2362-4d1c-be02-0e2c61671033 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:11.002668 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 770/1534] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:10 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:11.008137 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb461d-2f73-4e76-975a-e762c6a0c398 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:11.081188 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e07e0-2362-4d1c-be02-0e2c61671033 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:11.090665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb461d-2f73-4e76-975a-e762c6a0c398 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:11.092508 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e010a2-891c-4dad-965f-39969d08f0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:11.093930 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e010a2-891c-4dad-965f-39969d08f0bd admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:11.093930 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e010a2-891c-4dad-965f-39969d08f0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:11.094197 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e010a2-891c-4dad-965f-39969d08f0bd admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:11.095057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e010a2-891c-4dad-965f-39969d08f0bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:11.095498 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c7fb2-3a5f-4871-a185-69bca3cf37a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:11.096087 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c7fb2-3a5f-4871-a185-69bca3cf37a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:11.096591 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c7fb2-3a5f-4871-a185-69bca3cf37a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:11.096901 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2c7fb2-3a5f-4871-a185-69bca3cf37a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:11.098126 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e010a2-891c-4dad-965f-39969d08f0bd admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:11.098228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c7fb2-3a5f-4871-a185-69bca3cf37a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:11.100351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2c7fb2-3a5f-4871-a185-69bca3cf37a2 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:11.108829 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:11.127534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 765/1535] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:10 2022] POST /identity/v3/users => generated 289 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:11.136182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 771/1536] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:11 2022] POST /identity/v3/users => generated 295 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:36:11.136612 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e010a2-891c-4dad-965f-39969d08f0bd admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:11.137979 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c7fb2-3a5f-4871-a185-69bca3cf37a2 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:11.147534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e010a2-891c-4dad-965f-39969d08f0bd admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:11.153584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efef457-0858-4db4-856a-c56202eb0a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:11.154021 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2c7fb2-3a5f-4871-a185-69bca3cf37a2 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:11.154609 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efef457-0858-4db4-856a-c56202eb0a0d admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:11.154609 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efef457-0858-4db4-856a-c56202eb0a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:11.154940 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efef457-0858-4db4-856a-c56202eb0a0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:11.155579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efef457-0858-4db4-856a-c56202eb0a0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:11.157816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efef457-0858-4db4-856a-c56202eb0a0d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:11.164412 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f3d1b-9251-479d-85b1-240beb3ce29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:11.165270 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f3d1b-9251-479d-85b1-240beb3ce29a admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:11.165551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f3d1b-9251-479d-85b1-240beb3ce29a admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:11.165826 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f3d1b-9251-479d-85b1-240beb3ce29a admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:11.166804 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f3d1b-9251-479d-85b1-240beb3ce29a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:11.168805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f3d1b-9251-479d-85b1-240beb3ce29a admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:11.170286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 766/1537] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:11.174055 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efef457-0858-4db4-856a-c56202eb0a0d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:11.181154 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 772/1538] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:11.184816 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f3d1b-9251-479d-85b1-240beb3ce29a admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:11.187680 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efef457-0858-4db4-856a-c56202eb0a0d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:11.193819 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c381d-fda1-4410-ae96-ed5fdf5b68e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:11.194446 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c381d-fda1-4410-ae96-ed5fdf5b68e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:11.194732 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c381d-fda1-4410-ae96-ed5fdf5b68e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:11.195008 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c381d-fda1-4410-ae96-ed5fdf5b68e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:11.195655 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c381d-fda1-4410-ae96-ed5fdf5b68e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:11.196880 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f3d1b-9251-479d-85b1-240beb3ce29a admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:11.197298 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c381d-fda1-4410-ae96-ed5fdf5b68e2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:11.202239 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e524c-36e0-4e11-9596-64d341d962d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:11.203121 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e524c-36e0-4e11-9596-64d341d962d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:11.203397 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e524c-36e0-4e11-9596-64d341d962d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:11.203665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e524c-36e0-4e11-9596-64d341d962d4 admin admin] PATH_INFO: `/v3/projects/8db7452b4cf54f83a5df006baeac8372/users/61cdf43fc5e1423aa0ab9ba09dba9e2b/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:11.209699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 767/1539] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:36:11.213783 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c381d-fda1-4410-ae96-ed5fdf5b68e2 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:11.224402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c381d-fda1-4410-ae96-ed5fdf5b68e2 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:11.227252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e524c-36e0-4e11-9596-64d341d962d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8db7452b4cf54f83a5df006baeac8372, user_id=61cdf43fc5e1423aa0ab9ba09dba9e2b, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:11.231623 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e524c-36e0-4e11-9596-64d341d962d4 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:11.232235 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b739064-d0f9-4913-88a3-c970eabc6099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:11.233210 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b739064-d0f9-4913-88a3-c970eabc6099 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:11.233585 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b739064-d0f9-4913-88a3-c970eabc6099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:11.234282 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b739064-d0f9-4913-88a3-c970eabc6099 admin admin] PATH_INFO: `/v3/projects/fe5c694c6b584821b95eb97d68a83af1/users/933723f51b144c5081c8b280e3251d9f/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:11.245906 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 773/1540] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:11 2022] PUT /identity/v3/projects/8db7452b4cf54f83a5df006baeac8372/users/61cdf43fc5e1423aa0ab9ba09dba9e2b/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:36:11.249755 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e524c-36e0-4e11-9596-64d341d962d4 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:11.253423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b739064-d0f9-4913-88a3-c970eabc6099 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe5c694c6b584821b95eb97d68a83af1, user_id=933723f51b144c5081c8b280e3251d9f, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:11.256300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b739064-d0f9-4913-88a3-c970eabc6099 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:11.272488 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 768/1541] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:11 2022] PUT /identity/v3/projects/fe5c694c6b584821b95eb97d68a83af1/users/933723f51b144c5081c8b280e3251d9f/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:36:11.283478 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67359c9f-1f3f-4821-ac3d-2c21a5a0024d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:11.283766 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67359c9f-1f3f-4821-ac3d-2c21a5a0024d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:11.286648 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67359c9f-1f3f-4821-ac3d-2c21a5a0024d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:11.308063 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:11.314393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-67359c9f-1f3f-4821-ac3d-2c21a5a0024d None None] MFA Rules not processed for user `933723f51b144c5081c8b280e3251d9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:11.315644 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-67359c9f-1f3f-4821-ac3d-2c21a5a0024d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:11.324754 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e524c-36e0-4e11-9596-64d341d962d4 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:11.392551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b04d7-042d-4a04-8082-06599fb3481b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:11.393256 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b04d7-042d-4a04-8082-06599fb3481b admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:11.393537 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b04d7-042d-4a04-8082-06599fb3481b admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:11.393737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b04d7-042d-4a04-8082-06599fb3481b admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:11.394862 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887b04d7-042d-4a04-8082-06599fb3481b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:11.396514 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887b04d7-042d-4a04-8082-06599fb3481b admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:11.397215 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 769/1542] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:36:11.409096 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 774/1543] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:36:11.413923 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b04d7-042d-4a04-8082-06599fb3481b admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:11.414084 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67359c9f-1f3f-4821-ac3d-2c21a5a0024d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:11.424802 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b04d7-042d-4a04-8082-06599fb3481b admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:11.431769 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917245b6-7834-4151-a10b-36bc6b765528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:11.435083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917245b6-7834-4151-a10b-36bc6b765528 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:11.435083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917245b6-7834-4151-a10b-36bc6b765528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:11.435083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917245b6-7834-4151-a10b-36bc6b765528 admin admin] PATH_INFO: `/v3/projects/8db7452b4cf54f83a5df006baeac8372/users/61cdf43fc5e1423aa0ab9ba09dba9e2b/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:11.439538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917245b6-7834-4151-a10b-36bc6b765528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8db7452b4cf54f83a5df006baeac8372, user_id=61cdf43fc5e1423aa0ab9ba09dba9e2b, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:11.443170 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917245b6-7834-4151-a10b-36bc6b765528 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:11.464808 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 775/1544] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:11 2022] PUT /identity/v3/projects/8db7452b4cf54f83a5df006baeac8372/users/61cdf43fc5e1423aa0ab9ba09dba9e2b/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:11.467463 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8f296-62b4-4619-a5f7-3bcc8745cd04 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:11.467755 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8f296-62b4-4619-a5f7-3bcc8745cd04 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:11.468030 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8f296-62b4-4619-a5f7-3bcc8745cd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:11.500587 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-50e8f296-62b4-4619-a5f7-3bcc8745cd04 None None] MFA Rules not processed for user `61cdf43fc5e1423aa0ab9ba09dba9e2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:11.501974 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-50e8f296-62b4-4619-a5f7-3bcc8745cd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:11.521119 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67359c9f-1f3f-4821-ac3d-2c21a5a0024d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:11.583460 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:11.587727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b4bdb-ce55-4414-a8d9-bae8b2b1d4e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:11.588784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b4bdb-ce55-4414-a8d9-bae8b2b1d4e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:11.589193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b4bdb-ce55-4414-a8d9-bae8b2b1d4e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:11.589749 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b4bdb-ce55-4414-a8d9-bae8b2b1d4e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:11.662445 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 776/1545] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:11.673218 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e8f296-62b4-4619-a5f7-3bcc8745cd04 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:11.675093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-50e8f296-62b4-4619-a5f7-3bcc8745cd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:11.686803 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b4bdb-ce55-4414-a8d9-bae8b2b1d4e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:11.688908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b4bdb-ce55-4414-a8d9-bae8b2b1d4e0 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:11.692197 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e8f296-62b4-4619-a5f7-3bcc8745cd04 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:11.697925 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5552f-d5fe-4894-9668-3ffa162dbbb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:11.698617 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5552f-d5fe-4894-9668-3ffa162dbbb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:11.698969 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5552f-d5fe-4894-9668-3ffa162dbbb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:11.699316 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5552f-d5fe-4894-9668-3ffa162dbbb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:11.701237 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 770/1546] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:11.710918 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5552f-d5fe-4894-9668-3ffa162dbbb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:11.712683 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5552f-d5fe-4894-9668-3ffa162dbbb5 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:11.753489 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 777/1547] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:36:11.807953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b4bdb-ce55-4414-a8d9-bae8b2b1d4e0 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:11.823240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b4bdb-ce55-4414-a8d9-bae8b2b1d4e0 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:11.829046 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98a447-92a9-4db1-8846-2883230a78ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:11.829933 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98a447-92a9-4db1-8846-2883230a78ea service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:11.830537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98a447-92a9-4db1-8846-2883230a78ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:11.831352 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98a447-92a9-4db1-8846-2883230a78ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:11.862489 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7e98a447-92a9-4db1-8846-2883230a78ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:11.995492 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e98a447-92a9-4db1-8846-2883230a78ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:11.997006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e98a447-92a9-4db1-8846-2883230a78ea service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:12.007807 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 771/1548] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:21.720207 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=76,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:22.002262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=54 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:22.575151 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bff462-c18d-4548-b3f1-784ab8da271f None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:22.575689 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bff462-c18d-4548-b3f1-784ab8da271f None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:22.576204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bff462-c18d-4548-b3f1-784ab8da271f None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:22.578967 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 778/1549] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 20 13:36:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:36:22.588290 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a03ee5-1e4c-448c-b85a-0062391acd1f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:22.589050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a03ee5-1e4c-448c-b85a-0062391acd1f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:22.589442 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a03ee5-1e4c-448c-b85a-0062391acd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:22.618518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-85a03ee5-1e4c-448c-b85a-0062391acd1f None None] MFA Rules not processed for user `ee5c2a3855cb4a668177aa19f337778d`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:22.619755 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-85a03ee5-1e4c-448c-b85a-0062391acd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:22.707422 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bff462-c18d-4548-b3f1-784ab8da271f None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:22.719186 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 772/1550] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 20 13:36:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:22.766197 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a03ee5-1e4c-448c-b85a-0062391acd1f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:22.767852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-85a03ee5-1e4c-448c-b85a-0062391acd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:22.787955 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a03ee5-1e4c-448c-b85a-0062391acd1f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:22.793803 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad80e9-cb2e-436c-a114-aafba4cb3100 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:22.794552 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad80e9-cb2e-436c-a114-aafba4cb3100 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:22.794815 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad80e9-cb2e-436c-a114-aafba4cb3100 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:22.795107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad80e9-cb2e-436c-a114-aafba4cb3100 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:22.799446 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bff462-c18d-4548-b3f1-784ab8da271f None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:22.805396 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad80e9-cb2e-436c-a114-aafba4cb3100 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:22.805751 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e36568-aaf1-4343-a741-4ffe5cd9c4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:22.806468 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e36568-aaf1-4343-a741-4ffe5cd9c4c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:22.806814 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e36568-aaf1-4343-a741-4ffe5cd9c4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:22.806985 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad80e9-cb2e-436c-a114-aafba4cb3100 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:22.807162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e36568-aaf1-4343-a741-4ffe5cd9c4c2 admin admin] PATH_INFO: `/v3/users/94799edd30b74d169c40e8e497804816` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:22.808534 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e36568-aaf1-4343-a741-4ffe5cd9c4c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94799edd30b74d169c40e8e497804816)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:22.810581 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e36568-aaf1-4343-a741-4ffe5cd9c4c2 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:22.818590 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 773/1551] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:36:22.871413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-20e36568-aaf1-4343-a741-4ffe5cd9c4c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94799edd30b74d169c40e8e497804816'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:22.872095 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-20e36568-aaf1-4343-a741-4ffe5cd9c4c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94799edd30b74d169c40e8e497804816'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:22.879866 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-20e36568-aaf1-4343-a741-4ffe5cd9c4c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94799edd30b74d169c40e8e497804816'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:22.880603 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-20e36568-aaf1-4343-a741-4ffe5cd9c4c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94799edd30b74d169c40e8e497804816'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:22.887956 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-20e36568-aaf1-4343-a741-4ffe5cd9c4c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94799edd30b74d169c40e8e497804816'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:22.905338 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 779/1552] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:22 2022] DELETE /identity/v3/users/94799edd30b74d169c40e8e497804816 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:22.910271 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c1a6e-ed89-443d-b4a2-e565fad53499 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:22.910556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c1a6e-ed89-443d-b4a2-e565fad53499 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:22.910833 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c1a6e-ed89-443d-b4a2-e565fad53499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:22.929462 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d93c1a6e-ed89-443d-b4a2-e565fad53499 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:22.930648 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d93c1a6e-ed89-443d-b4a2-e565fad53499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:22.988979 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 774/1553] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:22.998484 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e36568-aaf1-4343-a741-4ffe5cd9c4c2 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:22.999933 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-20e36568-aaf1-4343-a741-4ffe5cd9c4c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:23.063051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e36568-aaf1-4343-a741-4ffe5cd9c4c2 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:23.067749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1f4df-1d81-43a8-9d7f-2fca5dd92ca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:23.068357 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1f4df-1d81-43a8-9d7f-2fca5dd92ca7 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:23.068635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1f4df-1d81-43a8-9d7f-2fca5dd92ca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:23.068936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1f4df-1d81-43a8-9d7f-2fca5dd92ca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:23.077936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a1f4df-1d81-43a8-9d7f-2fca5dd92ca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:23.079367 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a1f4df-1d81-43a8-9d7f-2fca5dd92ca7 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:23.089429 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 780/1554] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:23.468883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c1a6e-ed89-443d-b4a2-e565fad53499 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:23.470363 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d93c1a6e-ed89-443d-b4a2-e565fad53499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:23.485896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c1a6e-ed89-443d-b4a2-e565fad53499 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:23.492525 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e55107-5c31-4b3d-94d4-34e069f0d093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:23.492826 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e55107-5c31-4b3d-94d4-34e069f0d093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:23.493140 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e55107-5c31-4b3d-94d4-34e069f0d093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:23.493472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e55107-5c31-4b3d-94d4-34e069f0d093 admin admin] PATH_INFO: `/v3/projects/ec4cd4e97af548e68428ebadd9bfe51d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:23.508906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e55107-5c31-4b3d-94d4-34e069f0d093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec4cd4e97af548e68428ebadd9bfe51d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:23.508906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e55107-5c31-4b3d-94d4-34e069f0d093 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:23.558734 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-09e55107-5c31-4b3d-94d4-34e069f0d093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec4cd4e97af548e68428ebadd9bfe51d', 'request_id': 'req-09e55107-5c31-4b3d-94d4-34e069f0d093'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:23.564964 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-09e55107-5c31-4b3d-94d4-34e069f0d093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec4cd4e97af548e68428ebadd9bfe51d', 'request_id': 'req-09e55107-5c31-4b3d-94d4-34e069f0d093'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:23.572813 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-09e55107-5c31-4b3d-94d4-34e069f0d093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec4cd4e97af548e68428ebadd9bfe51d', 'request_id': 'req-09e55107-5c31-4b3d-94d4-34e069f0d093'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:23.583762 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-09e55107-5c31-4b3d-94d4-34e069f0d093 admin admin] The token cache is being invalidate because project ec4cd4e97af548e68428ebadd9bfe51d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:23.584052 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-09e55107-5c31-4b3d-94d4-34e069f0d093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:23.584706 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-09e55107-5c31-4b3d-94d4-34e069f0d093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:23.586239 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 775/1555] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:23 2022] DELETE /identity/v3/projects/ec4cd4e97af548e68428ebadd9bfe51d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:23.590249 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1f4df-1d81-43a8-9d7f-2fca5dd92ca7 service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:23.600850 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1f4df-1d81-43a8-9d7f-2fca5dd92ca7 service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:23.618598 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:23.660214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36991f-1087-476e-9af8-25315c83105c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:23.660993 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36991f-1087-476e-9af8-25315c83105c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:23.661283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36991f-1087-476e-9af8-25315c83105c admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:23.661556 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36991f-1087-476e-9af8-25315c83105c admin admin] PATH_INFO: `/v3/users/60108e21df4c4a128c12bbeb47a85c7d` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:23.662881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a36991f-1087-476e-9af8-25315c83105c admin admin] RBAC: Authorizing `identity:delete_user(user_id=60108e21df4c4a128c12bbeb47a85c7d)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:23.665143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a36991f-1087-476e-9af8-25315c83105c admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:23.718365 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-1a36991f-1087-476e-9af8-25315c83105c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60108e21df4c4a128c12bbeb47a85c7d'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:23.719040 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-1a36991f-1087-476e-9af8-25315c83105c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60108e21df4c4a128c12bbeb47a85c7d'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:23.726846 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-1a36991f-1087-476e-9af8-25315c83105c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60108e21df4c4a128c12bbeb47a85c7d'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:23.727505 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-1a36991f-1087-476e-9af8-25315c83105c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60108e21df4c4a128c12bbeb47a85c7d'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:23.735361 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-1a36991f-1087-476e-9af8-25315c83105c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60108e21df4c4a128c12bbeb47a85c7d'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:23.754671 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 781/1556] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:23 2022] DELETE /identity/v3/users/60108e21df4c4a128c12bbeb47a85c7d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:24.114540 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e55107-5c31-4b3d-94d4-34e069f0d093 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:24.116129 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-09e55107-5c31-4b3d-94d4-34e069f0d093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:24.126711 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:24.134069 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e55107-5c31-4b3d-94d4-34e069f0d093 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:24.192842 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927962c-9316-445c-988d-4cf21e21e66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:24.193596 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927962c-9316-445c-988d-4cf21e21e66f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:24.193872 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927962c-9316-445c-988d-4cf21e21e66f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:24.194140 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927962c-9316-445c-988d-4cf21e21e66f admin admin] PATH_INFO: `/v3/projects/ecb5d0913cc14a0c895f8b826bfdea4f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:24.195263 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927962c-9316-445c-988d-4cf21e21e66f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecb5d0913cc14a0c895f8b826bfdea4f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:24.197200 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927962c-9316-445c-988d-4cf21e21e66f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:24.242109 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3927962c-9316-445c-988d-4cf21e21e66f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecb5d0913cc14a0c895f8b826bfdea4f', 'request_id': 'req-3927962c-9316-445c-988d-4cf21e21e66f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:24.248113 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3927962c-9316-445c-988d-4cf21e21e66f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecb5d0913cc14a0c895f8b826bfdea4f', 'request_id': 'req-3927962c-9316-445c-988d-4cf21e21e66f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:24.256141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3927962c-9316-445c-988d-4cf21e21e66f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecb5d0913cc14a0c895f8b826bfdea4f', 'request_id': 'req-3927962c-9316-445c-988d-4cf21e21e66f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:24.266241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3927962c-9316-445c-988d-4cf21e21e66f admin admin] The token cache is being invalidate because project ecb5d0913cc14a0c895f8b826bfdea4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:24.266526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3927962c-9316-445c-988d-4cf21e21e66f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:24.267169 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-3927962c-9316-445c-988d-4cf21e21e66f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:24.268665 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 776/1557] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:24 2022] DELETE /identity/v3/projects/ecb5d0913cc14a0c895f8b826bfdea4f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:36:24.272510 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36991f-1087-476e-9af8-25315c83105c admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:24.274078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-1a36991f-1087-476e-9af8-25315c83105c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:24.286734 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36991f-1087-476e-9af8-25315c83105c admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:24.343851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8047dc-7be6-4b4a-a3be-1211b8e6ed49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:24.344489 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8047dc-7be6-4b4a-a3be-1211b8e6ed49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:24.344773 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8047dc-7be6-4b4a-a3be-1211b8e6ed49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:24.345080 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8047dc-7be6-4b4a-a3be-1211b8e6ed49 admin admin] PATH_INFO: `/v3/users/61cdf43fc5e1423aa0ab9ba09dba9e2b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:24.346207 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8047dc-7be6-4b4a-a3be-1211b8e6ed49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61cdf43fc5e1423aa0ab9ba09dba9e2b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:24.348265 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8047dc-7be6-4b4a-a3be-1211b8e6ed49 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:24.400939 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:24.403772 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-1e8047dc-7be6-4b4a-a3be-1211b8e6ed49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61cdf43fc5e1423aa0ab9ba09dba9e2b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:24.404534 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-1e8047dc-7be6-4b4a-a3be-1211b8e6ed49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61cdf43fc5e1423aa0ab9ba09dba9e2b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:24.416992 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-1e8047dc-7be6-4b4a-a3be-1211b8e6ed49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61cdf43fc5e1423aa0ab9ba09dba9e2b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:24.418172 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-1e8047dc-7be6-4b4a-a3be-1211b8e6ed49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61cdf43fc5e1423aa0ab9ba09dba9e2b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:24.429381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-1e8047dc-7be6-4b4a-a3be-1211b8e6ed49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61cdf43fc5e1423aa0ab9ba09dba9e2b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:24.449047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 782/1558] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:24 2022] DELETE /identity/v3/users/61cdf43fc5e1423aa0ab9ba09dba9e2b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:24.749985 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927962c-9316-445c-988d-4cf21e21e66f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:24.751473 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3927962c-9316-445c-988d-4cf21e21e66f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:24.765411 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927962c-9316-445c-988d-4cf21e21e66f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:24.821960 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01bd43-e0e7-4d09-afbd-0260d5c6b128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:24.822649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01bd43-e0e7-4d09-afbd-0260d5c6b128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:24.822934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01bd43-e0e7-4d09-afbd-0260d5c6b128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:24.823218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01bd43-e0e7-4d09-afbd-0260d5c6b128 admin admin] PATH_INFO: `/v3/projects/8db7452b4cf54f83a5df006baeac8372` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:24.824331 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01bd43-e0e7-4d09-afbd-0260d5c6b128 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8db7452b4cf54f83a5df006baeac8372)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:24.826300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01bd43-e0e7-4d09-afbd-0260d5c6b128 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:24.866234 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:24.881052 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-2e01bd43-e0e7-4d09-afbd-0260d5c6b128 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8db7452b4cf54f83a5df006baeac8372', 'request_id': 'req-2e01bd43-e0e7-4d09-afbd-0260d5c6b128'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:24.887591 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-2e01bd43-e0e7-4d09-afbd-0260d5c6b128 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8db7452b4cf54f83a5df006baeac8372', 'request_id': 'req-2e01bd43-e0e7-4d09-afbd-0260d5c6b128'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:24.895662 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-2e01bd43-e0e7-4d09-afbd-0260d5c6b128 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8db7452b4cf54f83a5df006baeac8372', 'request_id': 'req-2e01bd43-e0e7-4d09-afbd-0260d5c6b128'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:24.906068 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-2e01bd43-e0e7-4d09-afbd-0260d5c6b128 admin admin] The token cache is being invalidate because project 8db7452b4cf54f83a5df006baeac8372 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:24.906355 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-2e01bd43-e0e7-4d09-afbd-0260d5c6b128 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:24.906988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-2e01bd43-e0e7-4d09-afbd-0260d5c6b128 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:24.908541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 777/1559] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:24 2022] DELETE /identity/v3/projects/8db7452b4cf54f83a5df006baeac8372 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:24.915174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641f995-7619-4bf2-9ba3-e79df933469d None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:24.915460 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641f995-7619-4bf2-9ba3-e79df933469d None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:24.915732 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641f995-7619-4bf2-9ba3-e79df933469d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:24.935043 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-1641f995-7619-4bf2-9ba3-e79df933469d None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:24.936217 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-1641f995-7619-4bf2-9ba3-e79df933469d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:24.995688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 783/1560] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:36:25.005722 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b96648-53c0-4abe-a5be-be9b25a6cc92 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:25.006007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b96648-53c0-4abe-a5be-be9b25a6cc92 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:25.006278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b96648-53c0-4abe-a5be-be9b25a6cc92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:25.026734 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-01b96648-53c0-4abe-a5be-be9b25a6cc92 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:25.027954 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-01b96648-53c0-4abe-a5be-be9b25a6cc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:25.039654 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 778/1561] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:25.041330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641f995-7619-4bf2-9ba3-e79df933469d None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:25.051747 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641f995-7619-4bf2-9ba3-e79df933469d None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:25.057059 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248f7a1-345f-445a-881d-beb938257a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:25.057696 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248f7a1-345f-445a-881d-beb938257a61 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:25.057982 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248f7a1-345f-445a-881d-beb938257a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:25.058488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248f7a1-345f-445a-881d-beb938257a61 admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:25.059180 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6248f7a1-345f-445a-881d-beb938257a61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:36:25.060174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6248f7a1-345f-445a-881d-beb938257a61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:25.061485 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6248f7a1-345f-445a-881d-beb938257a61 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:25.086349 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 784/1562] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:36:25.086660 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b96648-53c0-4abe-a5be-be9b25a6cc92 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:25.106015 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b96648-53c0-4abe-a5be-be9b25a6cc92 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:25.119399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265d01b-e9f2-4317-96cf-fd72caed44f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:25.121872 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265d01b-e9f2-4317-96cf-fd72caed44f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:25.121872 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265d01b-e9f2-4317-96cf-fd72caed44f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:25.121872 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265d01b-e9f2-4317-96cf-fd72caed44f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:25.121872 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b265d01b-e9f2-4317-96cf-fd72caed44f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:25.127581 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b265d01b-e9f2-4317-96cf-fd72caed44f7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:25.214450 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 779/1563] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:25 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:25.216320 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248f7a1-345f-445a-881d-beb938257a61 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:25.238651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248f7a1-345f-445a-881d-beb938257a61 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:25.344840 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234e1a3-4976-45b5-baf1-9df7247f586d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:25.345509 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234e1a3-4976-45b5-baf1-9df7247f586d admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:25.345771 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234e1a3-4976-45b5-baf1-9df7247f586d admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:25.346240 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234e1a3-4976-45b5-baf1-9df7247f586d admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:25.346989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6234e1a3-4976-45b5-baf1-9df7247f586d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:25.349618 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6234e1a3-4976-45b5-baf1-9df7247f586d admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:25.387874 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265d01b-e9f2-4317-96cf-fd72caed44f7 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:25.397793 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 785/1564] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:25 2022] POST /identity/v3/users => generated 291 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:25.402076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265d01b-e9f2-4317-96cf-fd72caed44f7 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:25.408004 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d313d7-4b74-439e-bb9a-8502189f6078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:25.408719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d313d7-4b74-439e-bb9a-8502189f6078 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:25.409128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d313d7-4b74-439e-bb9a-8502189f6078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:25.409759 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d313d7-4b74-439e-bb9a-8502189f6078 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:25.410581 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d313d7-4b74-439e-bb9a-8502189f6078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:25.413138 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d313d7-4b74-439e-bb9a-8502189f6078 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:25.428032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 780/1565] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:25.432035 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234e1a3-4976-45b5-baf1-9df7247f586d admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:25.447967 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234e1a3-4976-45b5-baf1-9df7247f586d admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:25.459442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09636783-6257-45a0-b157-882959ce1c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:25.460220 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09636783-6257-45a0-b157-882959ce1c93 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:25.462241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09636783-6257-45a0-b157-882959ce1c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:25.462654 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09636783-6257-45a0-b157-882959ce1c93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:25.463902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09636783-6257-45a0-b157-882959ce1c93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:25.479348 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09636783-6257-45a0-b157-882959ce1c93 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:25.663317 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 786/1566] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:25.667778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d313d7-4b74-439e-bb9a-8502189f6078 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:25.685864 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d313d7-4b74-439e-bb9a-8502189f6078 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:25.695824 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ffe59-2cd4-42bf-8200-819bd4a2bad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:25.697112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ffe59-2cd4-42bf-8200-819bd4a2bad5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:25.697790 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ffe59-2cd4-42bf-8200-819bd4a2bad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:25.698506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ffe59-2cd4-42bf-8200-819bd4a2bad5 admin admin] PATH_INFO: `/v3/projects/ebef9cca39cf4d2a9c9a2a6cc87efa44/users/bf5d7d966d55444da52361834662ad33/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:25.730020 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ffe59-2cd4-42bf-8200-819bd4a2bad5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebef9cca39cf4d2a9c9a2a6cc87efa44, user_id=bf5d7d966d55444da52361834662ad33, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:25.734936 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ffe59-2cd4-42bf-8200-819bd4a2bad5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:25.784363 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa79d0-e63c-4eb8-880b-3015534f1910 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:25.785647 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa79d0-e63c-4eb8-880b-3015534f1910 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:25.786711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa79d0-e63c-4eb8-880b-3015534f1910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:25.791200 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 781/1567] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:25 2022] PUT /identity/v3/projects/ebef9cca39cf4d2a9c9a2a6cc87efa44/users/bf5d7d966d55444da52361834662ad33/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:36:25.823547 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-faaa79d0-e63c-4eb8-880b-3015534f1910 None None] MFA Rules not processed for user `bf5d7d966d55444da52361834662ad33`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:25.825214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-faaa79d0-e63c-4eb8-880b-3015534f1910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:25.842235 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:25.906863 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 787/1568] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:25.912315 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ffe59-2cd4-42bf-8200-819bd4a2bad5 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:26.015636 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ffe59-2cd4-42bf-8200-819bd4a2bad5 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:26.021212 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e633817-4cfd-4f90-9ee2-f5c1991ecd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:26.021824 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e633817-4cfd-4f90-9ee2-f5c1991ecd32 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:26.022107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e633817-4cfd-4f90-9ee2-f5c1991ecd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:26.022385 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e633817-4cfd-4f90-9ee2-f5c1991ecd32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:26.023032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e633817-4cfd-4f90-9ee2-f5c1991ecd32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:26.024967 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e633817-4cfd-4f90-9ee2-f5c1991ecd32 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:26.059256 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 782/1569] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:25 2022] POST /identity/v3/projects => generated 383 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:26.063343 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa79d0-e63c-4eb8-880b-3015534f1910 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:26.075706 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa79d0-e63c-4eb8-880b-3015534f1910 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:26.146839 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d56292-65d4-465b-9f4f-1a6180a98255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:26.147854 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d56292-65d4-465b-9f4f-1a6180a98255 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:26.148429 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d56292-65d4-465b-9f4f-1a6180a98255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:26.148718 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d56292-65d4-465b-9f4f-1a6180a98255 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:26.149449 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d56292-65d4-465b-9f4f-1a6180a98255 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:26.151423 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d56292-65d4-465b-9f4f-1a6180a98255 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:26.173358 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 788/1570] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:26 2022] POST /identity/v3/users => generated 298 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:26.177929 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e633817-4cfd-4f90-9ee2-f5c1991ecd32 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:26.190306 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e633817-4cfd-4f90-9ee2-f5c1991ecd32 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:26.195256 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae18f5d-3213-4fa9-bf25-608ce218b014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:26.195876 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae18f5d-3213-4fa9-bf25-608ce218b014 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:26.196219 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae18f5d-3213-4fa9-bf25-608ce218b014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:26.196505 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae18f5d-3213-4fa9-bf25-608ce218b014 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:26.197192 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae18f5d-3213-4fa9-bf25-608ce218b014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:26.198821 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae18f5d-3213-4fa9-bf25-608ce218b014 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:26.212903 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 783/1571] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:26.214437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d56292-65d4-465b-9f4f-1a6180a98255 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:26.227545 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d56292-65d4-465b-9f4f-1a6180a98255 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:26.234108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8231a-21c0-418d-92d0-d6b50e2d1d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:26.234939 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8231a-21c0-418d-92d0-d6b50e2d1d34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:26.235653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8231a-21c0-418d-92d0-d6b50e2d1d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:26.236013 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8231a-21c0-418d-92d0-d6b50e2d1d34 admin admin] PATH_INFO: `/v3/projects/380c2070914347f1a55046884b006a24/users/e3b528f3f44e4ff1959a42e738e0a10f/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:26.262533 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8231a-21c0-418d-92d0-d6b50e2d1d34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=380c2070914347f1a55046884b006a24, user_id=e3b528f3f44e4ff1959a42e738e0a10f, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:26.265941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8231a-21c0-418d-92d0-d6b50e2d1d34 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:26.284067 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 789/1572] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:26 2022] PUT /identity/v3/projects/380c2070914347f1a55046884b006a24/users/e3b528f3f44e4ff1959a42e738e0a10f/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:26.288344 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae18f5d-3213-4fa9-bf25-608ce218b014 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:26.300917 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae18f5d-3213-4fa9-bf25-608ce218b014 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:26.343324 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:26.375159 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb05be-2b67-405d-b052-c234d6369d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:26.375868 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb05be-2b67-405d-b052-c234d6369d5c admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:26.376240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb05be-2b67-405d-b052-c234d6369d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:26.376821 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb05be-2b67-405d-b052-c234d6369d5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:26.377669 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb05be-2b67-405d-b052-c234d6369d5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:26.381018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb05be-2b67-405d-b052-c234d6369d5c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:26.395013 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 784/1573] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:36:26.396131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8231a-21c0-418d-92d0-d6b50e2d1d34 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:26.406527 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8231a-21c0-418d-92d0-d6b50e2d1d34 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:26.411223 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bbcdeb-b3a2-497b-8b7e-6d80a0b1e1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:26.411920 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbcdeb-b3a2-497b-8b7e-6d80a0b1e1c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:26.412086 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbcdeb-b3a2-497b-8b7e-6d80a0b1e1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:26.412363 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbcdeb-b3a2-497b-8b7e-6d80a0b1e1c7 admin admin] PATH_INFO: `/v3/projects/380c2070914347f1a55046884b006a24/users/e3b528f3f44e4ff1959a42e738e0a10f/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:26.414264 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bbcdeb-b3a2-497b-8b7e-6d80a0b1e1c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=380c2070914347f1a55046884b006a24, user_id=e3b528f3f44e4ff1959a42e738e0a10f, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:26.416910 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bbcdeb-b3a2-497b-8b7e-6d80a0b1e1c7 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:26.428627 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 790/1574] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:26 2022] PUT /identity/v3/projects/380c2070914347f1a55046884b006a24/users/e3b528f3f44e4ff1959a42e738e0a10f/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:36:26.439406 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5d644-0d7c-41be-8f15-3bf9fbeed351 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:26.439727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5d644-0d7c-41be-8f15-3bf9fbeed351 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:26.440004 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5d644-0d7c-41be-8f15-3bf9fbeed351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:26.490036 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-58c5d644-0d7c-41be-8f15-3bf9fbeed351 None None] MFA Rules not processed for user `e3b528f3f44e4ff1959a42e738e0a10f`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:26.491475 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-58c5d644-0d7c-41be-8f15-3bf9fbeed351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:26.712351 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 785/1575] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:26.726463 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bbcdeb-b3a2-497b-8b7e-6d80a0b1e1c7 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:26.728376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-09bbcdeb-b3a2-497b-8b7e-6d80a0b1e1c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:26.817838 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bbcdeb-b3a2-497b-8b7e-6d80a0b1e1c7 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:26.821797 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f6140-67de-4a0d-a104-8a9b579d2e99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:26.822523 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f6140-67de-4a0d-a104-8a9b579d2e99 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:26.841257 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f6140-67de-4a0d-a104-8a9b579d2e99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:26.841257 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f6140-67de-4a0d-a104-8a9b579d2e99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:26.841257 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f6140-67de-4a0d-a104-8a9b579d2e99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:26.841257 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f6140-67de-4a0d-a104-8a9b579d2e99 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:26.850350 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 791/1576] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:26 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:26.953882 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5d644-0d7c-41be-8f15-3bf9fbeed351 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:26.964156 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5d644-0d7c-41be-8f15-3bf9fbeed351 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:26.969644 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d948ed1-c165-4ad8-8fe7-09a1fdb4a18d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:26.970600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d948ed1-c165-4ad8-8fe7-09a1fdb4a18d service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:26.970935 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d948ed1-c165-4ad8-8fe7-09a1fdb4a18d service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:26.971272 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d948ed1-c165-4ad8-8fe7-09a1fdb4a18d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:27.054448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d948ed1-c165-4ad8-8fe7-09a1fdb4a18d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:27.055735 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d948ed1-c165-4ad8-8fe7-09a1fdb4a18d service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:27.066044 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 786/1577] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:27.510403 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f6140-67de-4a0d-a104-8a9b579d2e99 service cinder] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:27.524006 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:27.600488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f6140-67de-4a0d-a104-8a9b579d2e99 service cinder] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:27.610148 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93f095-dcd2-4f6e-bb36-26ecaf917d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:27.611475 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93f095-dcd2-4f6e-bb36-26ecaf917d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:27.612038 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93f095-dcd2-4f6e-bb36-26ecaf917d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:27.614307 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93f095-dcd2-4f6e-bb36-26ecaf917d0e admin admin] PATH_INFO: `/v3/users/bf5d7d966d55444da52361834662ad33` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:27.615990 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93f095-dcd2-4f6e-bb36-26ecaf917d0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf5d7d966d55444da52361834662ad33)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:27.618802 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93f095-dcd2-4f6e-bb36-26ecaf917d0e admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:27.689714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0f93f095-dcd2-4f6e-bb36-26ecaf917d0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf5d7d966d55444da52361834662ad33'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:27.692250 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0f93f095-dcd2-4f6e-bb36-26ecaf917d0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf5d7d966d55444da52361834662ad33'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:27.704459 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0f93f095-dcd2-4f6e-bb36-26ecaf917d0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf5d7d966d55444da52361834662ad33'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:27.705480 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0f93f095-dcd2-4f6e-bb36-26ecaf917d0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf5d7d966d55444da52361834662ad33'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:27.716968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0f93f095-dcd2-4f6e-bb36-26ecaf917d0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf5d7d966d55444da52361834662ad33'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:27.744288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 792/1578] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:27 2022] DELETE /identity/v3/users/bf5d7d966d55444da52361834662ad33 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:27.748630 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d9d7f7-2784-4f13-a853-2840d7ada37e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:27.748965 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d9d7f7-2784-4f13-a853-2840d7ada37e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:27.749222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d9d7f7-2784-4f13-a853-2840d7ada37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:27.775771 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-54d9d7f7-2784-4f13-a853-2840d7ada37e None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:27.777209 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-54d9d7f7-2784-4f13-a853-2840d7ada37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:27.838346 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:27.853356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 787/1579] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:27.863703 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93f095-dcd2-4f6e-bb36-26ecaf917d0e admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:27.865370 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-0f93f095-dcd2-4f6e-bb36-26ecaf917d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:28.083272 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93f095-dcd2-4f6e-bb36-26ecaf917d0e admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:28.088351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b5702-07f0-471f-93b0-cf39518e66f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:28.089089 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b5702-07f0-471f-93b0-cf39518e66f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:28.089377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b5702-07f0-471f-93b0-cf39518e66f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:28.089654 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b5702-07f0-471f-93b0-cf39518e66f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:28.119624 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b5702-07f0-471f-93b0-cf39518e66f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:28.121360 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b5702-07f0-471f-93b0-cf39518e66f0 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:28.148572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 793/1580] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:28.888524 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d9d7f7-2784-4f13-a853-2840d7ada37e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:28.890124 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-54d9d7f7-2784-4f13-a853-2840d7ada37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:28.905513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d9d7f7-2784-4f13-a853-2840d7ada37e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:28.914048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46fb8bd-eb1d-47f7-b4dd-17f324177254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:28.914611 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fb8bd-eb1d-47f7-b4dd-17f324177254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:28.914897 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fb8bd-eb1d-47f7-b4dd-17f324177254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:28.915148 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fb8bd-eb1d-47f7-b4dd-17f324177254 admin admin] PATH_INFO: `/v3/projects/ebef9cca39cf4d2a9c9a2a6cc87efa44` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:28.922047 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46fb8bd-eb1d-47f7-b4dd-17f324177254 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebef9cca39cf4d2a9c9a2a6cc87efa44)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:28.923948 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46fb8bd-eb1d-47f7-b4dd-17f324177254 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:28.992497 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c46fb8bd-eb1d-47f7-b4dd-17f324177254 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebef9cca39cf4d2a9c9a2a6cc87efa44', 'request_id': 'req-c46fb8bd-eb1d-47f7-b4dd-17f324177254'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:28.999653 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c46fb8bd-eb1d-47f7-b4dd-17f324177254 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebef9cca39cf4d2a9c9a2a6cc87efa44', 'request_id': 'req-c46fb8bd-eb1d-47f7-b4dd-17f324177254'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:29.010422 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c46fb8bd-eb1d-47f7-b4dd-17f324177254 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebef9cca39cf4d2a9c9a2a6cc87efa44', 'request_id': 'req-c46fb8bd-eb1d-47f7-b4dd-17f324177254'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:29.029137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c46fb8bd-eb1d-47f7-b4dd-17f324177254 admin admin] The token cache is being invalidate because project ebef9cca39cf4d2a9c9a2a6cc87efa44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:29.029525 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c46fb8bd-eb1d-47f7-b4dd-17f324177254 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:29.030540 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c46fb8bd-eb1d-47f7-b4dd-17f324177254 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:29.036032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 788/1581] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:28 2022] DELETE /identity/v3/projects/ebef9cca39cf4d2a9c9a2a6cc87efa44 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:29.037941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b5702-07f0-471f-93b0-cf39518e66f0 service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:29.047770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b5702-07f0-471f-93b0-cf39518e66f0 service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:29.085543 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:29.121443 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231b106-c3dc-4a7a-b110-78bcc8ec516e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:29.122090 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231b106-c3dc-4a7a-b110-78bcc8ec516e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:29.124737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231b106-c3dc-4a7a-b110-78bcc8ec516e admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:29.124737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231b106-c3dc-4a7a-b110-78bcc8ec516e admin admin] PATH_INFO: `/v3/users/e3b528f3f44e4ff1959a42e738e0a10f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:29.125773 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0231b106-c3dc-4a7a-b110-78bcc8ec516e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3b528f3f44e4ff1959a42e738e0a10f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:29.128990 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0231b106-c3dc-4a7a-b110-78bcc8ec516e admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:29.202323 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0231b106-c3dc-4a7a-b110-78bcc8ec516e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3b528f3f44e4ff1959a42e738e0a10f'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:29.202858 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0231b106-c3dc-4a7a-b110-78bcc8ec516e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3b528f3f44e4ff1959a42e738e0a10f'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:29.213559 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0231b106-c3dc-4a7a-b110-78bcc8ec516e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3b528f3f44e4ff1959a42e738e0a10f'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:29.214325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0231b106-c3dc-4a7a-b110-78bcc8ec516e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3b528f3f44e4ff1959a42e738e0a10f'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:29.228244 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0231b106-c3dc-4a7a-b110-78bcc8ec516e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3b528f3f44e4ff1959a42e738e0a10f'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:29.260766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 794/1582] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:29 2022] DELETE /identity/v3/users/e3b528f3f44e4ff1959a42e738e0a10f => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:29.604792 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46fb8bd-eb1d-47f7-b4dd-17f324177254 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:29.606514 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c46fb8bd-eb1d-47f7-b4dd-17f324177254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:29.622859 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46fb8bd-eb1d-47f7-b4dd-17f324177254 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:29.717947 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c59e91-7e6f-45dd-8bc2-4e9f4298c785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:29.719218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c59e91-7e6f-45dd-8bc2-4e9f4298c785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:29.719642 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c59e91-7e6f-45dd-8bc2-4e9f4298c785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:29.720334 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c59e91-7e6f-45dd-8bc2-4e9f4298c785 admin admin] PATH_INFO: `/v3/projects/380c2070914347f1a55046884b006a24` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:29.722454 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c59e91-7e6f-45dd-8bc2-4e9f4298c785 admin admin] RBAC: Authorizing `identity:delete_project(project_id=380c2070914347f1a55046884b006a24)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:29.724618 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c59e91-7e6f-45dd-8bc2-4e9f4298c785 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:29.798914 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f9c59e91-7e6f-45dd-8bc2-4e9f4298c785 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '380c2070914347f1a55046884b006a24', 'request_id': 'req-f9c59e91-7e6f-45dd-8bc2-4e9f4298c785'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:29.806427 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f9c59e91-7e6f-45dd-8bc2-4e9f4298c785 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '380c2070914347f1a55046884b006a24', 'request_id': 'req-f9c59e91-7e6f-45dd-8bc2-4e9f4298c785'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:29.816932 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f9c59e91-7e6f-45dd-8bc2-4e9f4298c785 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '380c2070914347f1a55046884b006a24', 'request_id': 'req-f9c59e91-7e6f-45dd-8bc2-4e9f4298c785'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:29.830207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f9c59e91-7e6f-45dd-8bc2-4e9f4298c785 admin admin] The token cache is being invalidate because project 380c2070914347f1a55046884b006a24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:29.830505 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f9c59e91-7e6f-45dd-8bc2-4e9f4298c785 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:29.831709 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f9c59e91-7e6f-45dd-8bc2-4e9f4298c785 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:29.836564 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 789/1583] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:29 2022] DELETE /identity/v3/projects/380c2070914347f1a55046884b006a24 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:29.838547 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd4c1f-e125-40d5-8ade-57218eb9dc60 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:29.838839 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd4c1f-e125-40d5-8ade-57218eb9dc60 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:29.839121 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd4c1f-e125-40d5-8ade-57218eb9dc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:29.853961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c59e91-7e6f-45dd-8bc2-4e9f4298c785 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:29.855539 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f9c59e91-7e6f-45dd-8bc2-4e9f4298c785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:29.860019 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-96dd4c1f-e125-40d5-8ade-57218eb9dc60 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:29.861236 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-96dd4c1f-e125-40d5-8ade-57218eb9dc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:29.865966 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:29.926610 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 795/1584] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:29.934433 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c59e91-7e6f-45dd-8bc2-4e9f4298c785 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:29.935413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8fa112-2858-4e72-b8cc-f1231246e54d None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:29.935695 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8fa112-2858-4e72-b8cc-f1231246e54d None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:29.935963 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8fa112-2858-4e72-b8cc-f1231246e54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:29.939098 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40785da1-0c56-40ba-82b9-8e1618f6a18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:29.939741 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40785da1-0c56-40ba-82b9-8e1618f6a18d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:29.940010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40785da1-0c56-40ba-82b9-8e1618f6a18d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:29.940332 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40785da1-0c56-40ba-82b9-8e1618f6a18d admin admin] PATH_INFO: `/v3/users/bde7409927c14eacb88c209f1191c3e3` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:29.941490 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40785da1-0c56-40ba-82b9-8e1618f6a18d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bde7409927c14eacb88c209f1191c3e3)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:29.943432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40785da1-0c56-40ba-82b9-8e1618f6a18d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:29.955272 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-ce8fa112-2858-4e72-b8cc-f1231246e54d None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:29.956687 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-ce8fa112-2858-4e72-b8cc-f1231246e54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:29.964560 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 796/1585] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:36:29.968560 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8fa112-2858-4e72-b8cc-f1231246e54d None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:29.978515 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8fa112-2858-4e72-b8cc-f1231246e54d None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:29.983149 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1b86e-4cc9-4ace-bca0-b099fb18b493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:29.983732 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1b86e-4cc9-4ace-bca0-b099fb18b493 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:29.984025 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1b86e-4cc9-4ace-bca0-b099fb18b493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:29.984275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1b86e-4cc9-4ace-bca0-b099fb18b493 admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:29.984930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1b86e-4cc9-4ace-bca0-b099fb18b493 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:36:29.985230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1b86e-4cc9-4ace-bca0-b099fb18b493 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:29.986802 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1b86e-4cc9-4ace-bca0-b099fb18b493 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:29.992286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-40785da1-0c56-40ba-82b9-8e1618f6a18d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bde7409927c14eacb88c209f1191c3e3'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:29.993290 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-40785da1-0c56-40ba-82b9-8e1618f6a18d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bde7409927c14eacb88c209f1191c3e3'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:29.998420 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 797/1586] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:36:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:36:30.001943 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-40785da1-0c56-40ba-82b9-8e1618f6a18d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bde7409927c14eacb88c209f1191c3e3'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:30.002677 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1b86e-4cc9-4ace-bca0-b099fb18b493 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:30.005246 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-aef1b86e-4cc9-4ace-bca0-b099fb18b493 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:30.010607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-40785da1-0c56-40ba-82b9-8e1618f6a18d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bde7409927c14eacb88c209f1191c3e3'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:30.011336 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-40785da1-0c56-40ba-82b9-8e1618f6a18d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bde7409927c14eacb88c209f1191c3e3'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:30.017506 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1b86e-4cc9-4ace-bca0-b099fb18b493 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:30.022313 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86365d1-db90-43a8-82db-0f7fa730005f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:30.022859 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86365d1-db90-43a8-82db-0f7fa730005f admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:30.023116 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86365d1-db90-43a8-82db-0f7fa730005f admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:30.023384 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86365d1-db90-43a8-82db-0f7fa730005f admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:30.024003 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86365d1-db90-43a8-82db-0f7fa730005f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:30.025907 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86365d1-db90-43a8-82db-0f7fa730005f admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:30.029275 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 790/1587] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:29 2022] DELETE /identity/v3/users/bde7409927c14eacb88c209f1191c3e3 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:30.033562 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768719fb-a401-4136-9483-a338d3d03fd1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:30.033850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768719fb-a401-4136-9483-a338d3d03fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:30.034645 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768719fb-a401-4136-9483-a338d3d03fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:30.053579 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 798/1588] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:30 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:36:30.054262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-768719fb-a401-4136-9483-a338d3d03fd1 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:30.055285 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-768719fb-a401-4136-9483-a338d3d03fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:30.057724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86365d1-db90-43a8-82db-0f7fa730005f admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:30.072123 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:30.120650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 791/1589] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:30.129345 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86365d1-db90-43a8-82db-0f7fa730005f admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:30.130760 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768719fb-a401-4136-9483-a338d3d03fd1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:30.132139 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-768719fb-a401-4136-9483-a338d3d03fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:30.134305 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad860d-1cec-490e-8ef2-d878a4df1866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:30.134997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad860d-1cec-490e-8ef2-d878a4df1866 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:30.135356 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad860d-1cec-490e-8ef2-d878a4df1866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:30.135754 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad860d-1cec-490e-8ef2-d878a4df1866 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:30.136361 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad860d-1cec-490e-8ef2-d878a4df1866 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:30.138455 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad860d-1cec-490e-8ef2-d878a4df1866 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:30.165794 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 799/1590] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:30 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:30.167316 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:30.170108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad860d-1cec-490e-8ef2-d878a4df1866 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:30.180191 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad860d-1cec-490e-8ef2-d878a4df1866 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:30.184800 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2300b2c-61c1-4d46-b15e-6db05bdb1466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:30.185431 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2300b2c-61c1-4d46-b15e-6db05bdb1466 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:30.185705 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2300b2c-61c1-4d46-b15e-6db05bdb1466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:30.185976 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2300b2c-61c1-4d46-b15e-6db05bdb1466 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:30.186610 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2300b2c-61c1-4d46-b15e-6db05bdb1466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:30.188185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2300b2c-61c1-4d46-b15e-6db05bdb1466 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:30.196182 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768719fb-a401-4136-9483-a338d3d03fd1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:30.198140 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 800/1591] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:30.201925 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2300b2c-61c1-4d46-b15e-6db05bdb1466 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:30.203205 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6679664-9c35-4258-932e-02ceb161434e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:30.203822 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6679664-9c35-4258-932e-02ceb161434e service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:30.204104 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6679664-9c35-4258-932e-02ceb161434e service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:30.204380 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6679664-9c35-4258-932e-02ceb161434e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:30.212993 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2300b2c-61c1-4d46-b15e-6db05bdb1466 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:30.214297 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6679664-9c35-4258-932e-02ceb161434e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:30.215727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6679664-9c35-4258-932e-02ceb161434e service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:30.217761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1156a-59dc-457e-b7c5-0811a2ae7ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:30.218372 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1156a-59dc-457e-b7c5-0811a2ae7ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:30.218682 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1156a-59dc-457e-b7c5-0811a2ae7ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:30.219089 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1156a-59dc-457e-b7c5-0811a2ae7ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:30.219724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1156a-59dc-457e-b7c5-0811a2ae7ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:30.221395 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1156a-59dc-457e-b7c5-0811a2ae7ea7 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:30.225986 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 792/1592] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:30.236397 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 801/1593] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:36:30.237703 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6679664-9c35-4258-932e-02ceb161434e service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:30.247102 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6679664-9c35-4258-932e-02ceb161434e service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:30.251742 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd89a09-ae30-44d0-ab8d-3e87a56fdc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:30.252393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd89a09-ae30-44d0-ab8d-3e87a56fdc62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:30.252674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd89a09-ae30-44d0-ab8d-3e87a56fdc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:30.253280 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd89a09-ae30-44d0-ab8d-3e87a56fdc62 admin admin] PATH_INFO: `/v3/projects/22424a637585415d86db35e23e1e06fe/users/a9e063603e0544468f8f610c8b3a52ee/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:30.270806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd89a09-ae30-44d0-ab8d-3e87a56fdc62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22424a637585415d86db35e23e1e06fe, user_id=a9e063603e0544468f8f610c8b3a52ee, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:30.273947 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd89a09-ae30-44d0-ab8d-3e87a56fdc62 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:30.289829 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 793/1594] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:30 2022] PUT /identity/v3/projects/22424a637585415d86db35e23e1e06fe/users/a9e063603e0544468f8f610c8b3a52ee/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:30.295296 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41e673-fa2e-477a-9195-49ac53d0f3ba None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:30.295589 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41e673-fa2e-477a-9195-49ac53d0f3ba None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:30.295861 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41e673-fa2e-477a-9195-49ac53d0f3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:30.316849 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-1a41e673-fa2e-477a-9195-49ac53d0f3ba None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:30.318087 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-1a41e673-fa2e-477a-9195-49ac53d0f3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:30.388448 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 802/1595] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:30.401499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd89a09-ae30-44d0-ab8d-3e87a56fdc62 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:30.477712 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd89a09-ae30-44d0-ab8d-3e87a56fdc62 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:30.483186 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e902c46-32d3-4cb9-b2c8-f1ef82dc725b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:30.483550 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e902c46-32d3-4cb9-b2c8-f1ef82dc725b service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:30.483917 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e902c46-32d3-4cb9-b2c8-f1ef82dc725b service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:30.484239 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e902c46-32d3-4cb9-b2c8-f1ef82dc725b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:30.494867 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e902c46-32d3-4cb9-b2c8-f1ef82dc725b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:30.502156 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e902c46-32d3-4cb9-b2c8-f1ef82dc725b service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:30.513158 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 794/1596] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:36:30.547161 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41e673-fa2e-477a-9195-49ac53d0f3ba None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:30.548596 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-1a41e673-fa2e-477a-9195-49ac53d0f3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:30.568456 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41e673-fa2e-477a-9195-49ac53d0f3ba None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:30.575658 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36670d-21f6-4658-b208-65d051b2ce57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:30.576406 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36670d-21f6-4658-b208-65d051b2ce57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:30.576741 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36670d-21f6-4658-b208-65d051b2ce57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:30.577994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36670d-21f6-4658-b208-65d051b2ce57 admin admin] PATH_INFO: `/v3/projects/9c119266a2814a0d987a6215b980065f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:30.578600 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36670d-21f6-4658-b208-65d051b2ce57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c119266a2814a0d987a6215b980065f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:30.580519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36670d-21f6-4658-b208-65d051b2ce57 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:30.645002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-fc36670d-21f6-4658-b208-65d051b2ce57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c119266a2814a0d987a6215b980065f', 'request_id': 'req-fc36670d-21f6-4658-b208-65d051b2ce57'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:30.656126 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-fc36670d-21f6-4658-b208-65d051b2ce57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c119266a2814a0d987a6215b980065f', 'request_id': 'req-fc36670d-21f6-4658-b208-65d051b2ce57'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:30.666322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-fc36670d-21f6-4658-b208-65d051b2ce57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c119266a2814a0d987a6215b980065f', 'request_id': 'req-fc36670d-21f6-4658-b208-65d051b2ce57'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:30.679506 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-fc36670d-21f6-4658-b208-65d051b2ce57 admin admin] The token cache is being invalidate because project 9c119266a2814a0d987a6215b980065f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:30.679795 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-fc36670d-21f6-4658-b208-65d051b2ce57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:30.680505 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-fc36670d-21f6-4658-b208-65d051b2ce57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:30.682045 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 803/1597] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:30 2022] DELETE /identity/v3/projects/9c119266a2814a0d987a6215b980065f => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:30.686206 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e902c46-32d3-4cb9-b2c8-f1ef82dc725b service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:30.698792 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e902c46-32d3-4cb9-b2c8-f1ef82dc725b service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:30.772610 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402b44e-8a7c-4d1b-8cad-1b9525d26969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:30.773692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402b44e-8a7c-4d1b-8cad-1b9525d26969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:30.774075 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402b44e-8a7c-4d1b-8cad-1b9525d26969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:30.774542 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402b44e-8a7c-4d1b-8cad-1b9525d26969 admin admin] PATH_INFO: `/v3/users/25d47307ee494cb3b0dba891fd194c50` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:30.775837 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402b44e-8a7c-4d1b-8cad-1b9525d26969 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25d47307ee494cb3b0dba891fd194c50)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:30.778777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402b44e-8a7c-4d1b-8cad-1b9525d26969 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:30.842610 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9402b44e-8a7c-4d1b-8cad-1b9525d26969 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25d47307ee494cb3b0dba891fd194c50'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:30.843369 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9402b44e-8a7c-4d1b-8cad-1b9525d26969 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25d47307ee494cb3b0dba891fd194c50'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:30.845990 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:30.858902 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9402b44e-8a7c-4d1b-8cad-1b9525d26969 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25d47307ee494cb3b0dba891fd194c50'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:30.872329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9402b44e-8a7c-4d1b-8cad-1b9525d26969 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25d47307ee494cb3b0dba891fd194c50'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:30.873319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9402b44e-8a7c-4d1b-8cad-1b9525d26969 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25d47307ee494cb3b0dba891fd194c50'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:30.904228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 795/1598] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:30 2022] DELETE /identity/v3/users/25d47307ee494cb3b0dba891fd194c50 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:31.910288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36670d-21f6-4658-b208-65d051b2ce57 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:31.912291 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-fc36670d-21f6-4658-b208-65d051b2ce57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:31.921237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:31.934737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36670d-21f6-4658-b208-65d051b2ce57 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:32.041273 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8050e-ce5c-4a05-9b23-6f3e44922520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:32.041932 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8050e-ce5c-4a05-9b23-6f3e44922520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:32.042204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8050e-ce5c-4a05-9b23-6f3e44922520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:32.042486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8050e-ce5c-4a05-9b23-6f3e44922520 admin admin] PATH_INFO: `/v3/projects/8adaf78b1f9f4d21a853a2cc14a26788` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:32.043614 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee8050e-ce5c-4a05-9b23-6f3e44922520 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8adaf78b1f9f4d21a853a2cc14a26788)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:32.045651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee8050e-ce5c-4a05-9b23-6f3e44922520 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:32.101357 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0ee8050e-ce5c-4a05-9b23-6f3e44922520 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8adaf78b1f9f4d21a853a2cc14a26788', 'request_id': 'req-0ee8050e-ce5c-4a05-9b23-6f3e44922520'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:32.112972 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0ee8050e-ce5c-4a05-9b23-6f3e44922520 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8adaf78b1f9f4d21a853a2cc14a26788', 'request_id': 'req-0ee8050e-ce5c-4a05-9b23-6f3e44922520'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:32.131681 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0ee8050e-ce5c-4a05-9b23-6f3e44922520 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8adaf78b1f9f4d21a853a2cc14a26788', 'request_id': 'req-0ee8050e-ce5c-4a05-9b23-6f3e44922520'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:32.173138 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0ee8050e-ce5c-4a05-9b23-6f3e44922520 admin admin] The token cache is being invalidate because project 8adaf78b1f9f4d21a853a2cc14a26788 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:32.177733 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0ee8050e-ce5c-4a05-9b23-6f3e44922520 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:32.180352 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0ee8050e-ce5c-4a05-9b23-6f3e44922520 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:32.190451 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bc4c8-1681-4b03-bfb0-c93674dac06b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:32.191387 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 804/1599] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:31 2022] DELETE /identity/v3/projects/8adaf78b1f9f4d21a853a2cc14a26788 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:32.191706 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bc4c8-1681-4b03-bfb0-c93674dac06b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:32.192126 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bc4c8-1681-4b03-bfb0-c93674dac06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:32.193443 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7dbbe3-cc77-4f00-bb55-7b0a6cb31e21 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:32.198900 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7dbbe3-cc77-4f00-bb55-7b0a6cb31e21 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:32.199245 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7dbbe3-cc77-4f00-bb55-7b0a6cb31e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:32.239000 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-672bc4c8-1681-4b03-bfb0-c93674dac06b None None] MFA Rules not processed for user `a9e063603e0544468f8f610c8b3a52ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:32.241306 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-672bc4c8-1681-4b03-bfb0-c93674dac06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:32.270741 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-0d7dbbe3-cc77-4f00-bb55-7b0a6cb31e21 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:32.274636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-0d7dbbe3-cc77-4f00-bb55-7b0a6cb31e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:32.365053 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 796/1600] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:36:32.365971 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bc4c8-1681-4b03-bfb0-c93674dac06b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:32.367844 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-672bc4c8-1681-4b03-bfb0-c93674dac06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:32.437223 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 805/1601] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:32.442453 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83013612-3309-4ba2-bdd1-2cfc961edab9 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:32.443590 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83013612-3309-4ba2-bdd1-2cfc961edab9 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:32.443976 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83013612-3309-4ba2-bdd1-2cfc961edab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:32.514451 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-83013612-3309-4ba2-bdd1-2cfc961edab9 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:32.520003 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-83013612-3309-4ba2-bdd1-2cfc961edab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:32.555847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 806/1602] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:32.559878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bc4c8-1681-4b03-bfb0-c93674dac06b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:32.560705 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83013612-3309-4ba2-bdd1-2cfc961edab9 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:32.577180 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911cfcb-4f33-4319-80ab-789775e5a88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:32.577819 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911cfcb-4f33-4319-80ab-789775e5a88d admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:32.578115 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911cfcb-4f33-4319-80ab-789775e5a88d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:32.578367 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911cfcb-4f33-4319-80ab-789775e5a88d admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:32.579139 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911cfcb-4f33-4319-80ab-789775e5a88d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:32.582829 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911cfcb-4f33-4319-80ab-789775e5a88d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:32.598799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83013612-3309-4ba2-bdd1-2cfc961edab9 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:32.620879 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855e5ce-420e-47ef-a98b-e7f78197ea41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:32.631213 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855e5ce-420e-47ef-a98b-e7f78197ea41 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:32.633230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855e5ce-420e-47ef-a98b-e7f78197ea41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:32.635168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855e5ce-420e-47ef-a98b-e7f78197ea41 admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:32.636019 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855e5ce-420e-47ef-a98b-e7f78197ea41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:36:32.636517 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855e5ce-420e-47ef-a98b-e7f78197ea41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:32.638458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855e5ce-420e-47ef-a98b-e7f78197ea41 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:32.657577 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 797/1603] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:32 2022] POST /identity/v3/projects => generated 371 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:32.661481 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911cfcb-4f33-4319-80ab-789775e5a88d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:32.671865 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 807/1604] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:36:32.682859 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855e5ce-420e-47ef-a98b-e7f78197ea41 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:32.699204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855e5ce-420e-47ef-a98b-e7f78197ea41 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:32.724339 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:32.827282 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb323474-b9a4-4f88-a0b4-b97adc85a481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:32.827909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb323474-b9a4-4f88-a0b4-b97adc85a481 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:32.828187 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb323474-b9a4-4f88-a0b4-b97adc85a481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:32.828456 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb323474-b9a4-4f88-a0b4-b97adc85a481 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:32.829156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb323474-b9a4-4f88-a0b4-b97adc85a481 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:32.831763 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:32.832024 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb323474-b9a4-4f88-a0b4-b97adc85a481 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:32.877343 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911cfcb-4f33-4319-80ab-789775e5a88d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:32.881793 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 808/1605] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:32 2022] POST /identity/v3/projects => generated 383 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:32.884006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054885e3-a1b5-48c0-93bd-f44dd848ee60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:32.884730 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054885e3-a1b5-48c0-93bd-f44dd848ee60 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:32.885392 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054885e3-a1b5-48c0-93bd-f44dd848ee60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:32.885767 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054885e3-a1b5-48c0-93bd-f44dd848ee60 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:32.886397 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb323474-b9a4-4f88-a0b4-b97adc85a481 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:32.886622 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054885e3-a1b5-48c0-93bd-f44dd848ee60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:32.888643 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054885e3-a1b5-48c0-93bd-f44dd848ee60 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:32.920794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 798/1606] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:32 2022] POST /identity/v3/users => generated 292 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:32.926374 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054885e3-a1b5-48c0-93bd-f44dd848ee60 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:32.939453 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054885e3-a1b5-48c0-93bd-f44dd848ee60 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:32.980185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb323474-b9a4-4f88-a0b4-b97adc85a481 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:32.985357 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598161b7-f7d9-4708-8702-df8a9220c511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:32.985981 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598161b7-f7d9-4708-8702-df8a9220c511 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:32.986260 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598161b7-f7d9-4708-8702-df8a9220c511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:32.986531 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598161b7-f7d9-4708-8702-df8a9220c511 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:32.987485 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598161b7-f7d9-4708-8702-df8a9220c511 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:32.989745 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598161b7-f7d9-4708-8702-df8a9220c511 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:33.007613 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c410072-7328-4ac4-be00-d42cd47aa5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:33.008688 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c410072-7328-4ac4-be00-d42cd47aa5fe admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:33.009270 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c410072-7328-4ac4-be00-d42cd47aa5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:33.009791 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c410072-7328-4ac4-be00-d42cd47aa5fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:33.010722 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c410072-7328-4ac4-be00-d42cd47aa5fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:33.012749 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c410072-7328-4ac4-be00-d42cd47aa5fe admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:33.027620 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 809/1607] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:32 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:36:33.031472 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598161b7-f7d9-4708-8702-df8a9220c511 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:33.037941 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 799/1608] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:33.038879 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c410072-7328-4ac4-be00-d42cd47aa5fe admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:33.058188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598161b7-f7d9-4708-8702-df8a9220c511 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:33.066719 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5e93e-5b42-4309-a2e7-1e7c2ac6631b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:33.067689 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5e93e-5b42-4309-a2e7-1e7c2ac6631b admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:33.067797 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c410072-7328-4ac4-be00-d42cd47aa5fe admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:33.068258 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5e93e-5b42-4309-a2e7-1e7c2ac6631b admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:33.069640 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5e93e-5b42-4309-a2e7-1e7c2ac6631b admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:33.070782 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5e93e-5b42-4309-a2e7-1e7c2ac6631b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:33.076478 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5e93e-5b42-4309-a2e7-1e7c2ac6631b admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:33.079848 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5ecd0-b6af-42c3-82ce-e02a080a2b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:33.080621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5ecd0-b6af-42c3-82ce-e02a080a2b4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:33.081463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5ecd0-b6af-42c3-82ce-e02a080a2b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:33.081713 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5ecd0-b6af-42c3-82ce-e02a080a2b4c admin admin] PATH_INFO: `/v3/projects/97ad6ebdac2d4e13871d268804f3340f/users/4e6fe9efa7f8487b96370ab4f99dc21c/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:33.099616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 810/1609] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:33.110983 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e5ecd0-b6af-42c3-82ce-e02a080a2b4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=97ad6ebdac2d4e13871d268804f3340f, user_id=4e6fe9efa7f8487b96370ab4f99dc21c, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:33.114554 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e5ecd0-b6af-42c3-82ce-e02a080a2b4c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:33.116822 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5e93e-5b42-4309-a2e7-1e7c2ac6631b admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:33.134841 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 800/1610] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:33 2022] PUT /identity/v3/projects/97ad6ebdac2d4e13871d268804f3340f/users/4e6fe9efa7f8487b96370ab4f99dc21c/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:33.139241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5ecd0-b6af-42c3-82ce-e02a080a2b4c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:33.147553 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5e93e-5b42-4309-a2e7-1e7c2ac6631b admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:33.255377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12779b-87bf-439d-b0b2-991d821a01e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:33.256272 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12779b-87bf-439d-b0b2-991d821a01e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:33.256735 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12779b-87bf-439d-b0b2-991d821a01e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:33.257138 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12779b-87bf-439d-b0b2-991d821a01e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:33.257889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12779b-87bf-439d-b0b2-991d821a01e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:33.260253 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12779b-87bf-439d-b0b2-991d821a01e5 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:33.275501 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 811/1611] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:33.279691 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12779b-87bf-439d-b0b2-991d821a01e5 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:33.283079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5ecd0-b6af-42c3-82ce-e02a080a2b4c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:33.300910 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ac951-05a5-41de-aa3a-010baeb8a1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:33.301160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12779b-87bf-439d-b0b2-991d821a01e5 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:33.301248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ac951-05a5-41de-aa3a-010baeb8a1fd admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:33.301248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ac951-05a5-41de-aa3a-010baeb8a1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:33.301248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ac951-05a5-41de-aa3a-010baeb8a1fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:33.301248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ac951-05a5-41de-aa3a-010baeb8a1fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:33.301248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ac951-05a5-41de-aa3a-010baeb8a1fd admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:33.301609 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b135a-fef7-4a6e-a7a1-476c16bac8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:33.301609 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b135a-fef7-4a6e-a7a1-476c16bac8b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:33.301609 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b135a-fef7-4a6e-a7a1-476c16bac8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:33.301609 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b135a-fef7-4a6e-a7a1-476c16bac8b4 admin admin] PATH_INFO: `/v3/projects/b272e5b187874e73b3356ec14840027b/users/2b2f3f7965e54830a074dbde9031220b/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:33.311135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 801/1612] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:33.312750 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:33.313477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ac951-05a5-41de-aa3a-010baeb8a1fd admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:33.320060 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b135a-fef7-4a6e-a7a1-476c16bac8b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b272e5b187874e73b3356ec14840027b, user_id=2b2f3f7965e54830a074dbde9031220b, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:33.323076 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b135a-fef7-4a6e-a7a1-476c16bac8b4 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:33.325875 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ac951-05a5-41de-aa3a-010baeb8a1fd admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:33.332058 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ca47b-8ba9-4752-8a30-0ad64b1ef583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:33.332854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ca47b-8ba9-4752-8a30-0ad64b1ef583 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:33.333268 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ca47b-8ba9-4752-8a30-0ad64b1ef583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:33.333644 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ca47b-8ba9-4752-8a30-0ad64b1ef583 admin admin] PATH_INFO: `/v3/projects/97ad6ebdac2d4e13871d268804f3340f/users/4e6fe9efa7f8487b96370ab4f99dc21c/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:33.335886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ca47b-8ba9-4752-8a30-0ad64b1ef583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97ad6ebdac2d4e13871d268804f3340f, user_id=4e6fe9efa7f8487b96370ab4f99dc21c, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:33.337160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 812/1613] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:33 2022] PUT /identity/v3/projects/b272e5b187874e73b3356ec14840027b/users/2b2f3f7965e54830a074dbde9031220b/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:33.342577 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ca47b-8ba9-4752-8a30-0ad64b1ef583 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:33.348599 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9b559-d13c-48a8-aedd-c5e0dccbb213 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:33.348988 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9b559-d13c-48a8-aedd-c5e0dccbb213 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:33.349271 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9b559-d13c-48a8-aedd-c5e0dccbb213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:33.355342 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 802/1614] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:33 2022] PUT /identity/v3/projects/97ad6ebdac2d4e13871d268804f3340f/users/4e6fe9efa7f8487b96370ab4f99dc21c/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:33.390211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-daa9b559-d13c-48a8-aedd-c5e0dccbb213 None None] MFA Rules not processed for user `2b2f3f7965e54830a074dbde9031220b`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:33.391807 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-daa9b559-d13c-48a8-aedd-c5e0dccbb213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:33.479232 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 813/1615] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:36:33.483835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ca47b-8ba9-4752-8a30-0ad64b1ef583 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:33.531526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:33.590391 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ca47b-8ba9-4752-8a30-0ad64b1ef583 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:33.595434 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9ff6e-dfbc-4a4b-9a45-1afe23ce83fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:33.596056 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9ff6e-dfbc-4a4b-9a45-1afe23ce83fe admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:33.596874 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9ff6e-dfbc-4a4b-9a45-1afe23ce83fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:33.597311 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9ff6e-dfbc-4a4b-9a45-1afe23ce83fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:33.597956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff9ff6e-dfbc-4a4b-9a45-1afe23ce83fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:33.599867 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff9ff6e-dfbc-4a4b-9a45-1afe23ce83fe admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:33.641039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 803/1616] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:33 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:36:33.642581 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9b559-d13c-48a8-aedd-c5e0dccbb213 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:33.658323 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9b559-d13c-48a8-aedd-c5e0dccbb213 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:33.767109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26486b37-47cd-44fc-823b-9d0067b9a103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:33.768130 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26486b37-47cd-44fc-823b-9d0067b9a103 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:33.768480 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26486b37-47cd-44fc-823b-9d0067b9a103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:33.768760 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26486b37-47cd-44fc-823b-9d0067b9a103 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:33.769482 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26486b37-47cd-44fc-823b-9d0067b9a103 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:33.771552 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26486b37-47cd-44fc-823b-9d0067b9a103 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:33.816346 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 814/1617] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:33 2022] POST /identity/v3/users => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:33.818483 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9ff6e-dfbc-4a4b-9a45-1afe23ce83fe admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:33.833818 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9ff6e-dfbc-4a4b-9a45-1afe23ce83fe admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:33.838994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52abc9-2c31-48da-b4bc-613fa38ea64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:33.839622 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52abc9-2c31-48da-b4bc-613fa38ea64b admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:33.839907 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52abc9-2c31-48da-b4bc-613fa38ea64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:33.840185 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52abc9-2c31-48da-b4bc-613fa38ea64b admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:33.840835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52abc9-2c31-48da-b4bc-613fa38ea64b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:33.842677 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52abc9-2c31-48da-b4bc-613fa38ea64b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:33.857682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 804/1618] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:36:33.859563 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26486b37-47cd-44fc-823b-9d0067b9a103 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:33.873835 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26486b37-47cd-44fc-823b-9d0067b9a103 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:33.881761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5834acb-7b3b-4629-b878-fb3195c33227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:33.882836 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5834acb-7b3b-4629-b878-fb3195c33227 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:33.883129 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5834acb-7b3b-4629-b878-fb3195c33227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:33.883479 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5834acb-7b3b-4629-b878-fb3195c33227 admin admin] PATH_INFO: `/v3/projects/c7755054241d44adb6c8f302f1e67fe9/users/643e03792f99411b82afc1fe5e2250d5/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:33.921348 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5834acb-7b3b-4629-b878-fb3195c33227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7755054241d44adb6c8f302f1e67fe9, user_id=643e03792f99411b82afc1fe5e2250d5, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:33.925421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5834acb-7b3b-4629-b878-fb3195c33227 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:33.946475 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 815/1619] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:33 2022] PUT /identity/v3/projects/c7755054241d44adb6c8f302f1e67fe9/users/643e03792f99411b82afc1fe5e2250d5/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:36:33.950695 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52abc9-2c31-48da-b4bc-613fa38ea64b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:33.962447 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52abc9-2c31-48da-b4bc-613fa38ea64b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:34.035443 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57e5f0-0130-42d8-b4d7-b40c18802c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:34.036075 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57e5f0-0130-42d8-b4d7-b40c18802c55 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:34.036634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57e5f0-0130-42d8-b4d7-b40c18802c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:34.036634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57e5f0-0130-42d8-b4d7-b40c18802c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:34.037370 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57e5f0-0130-42d8-b4d7-b40c18802c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:34.039118 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57e5f0-0130-42d8-b4d7-b40c18802c55 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:34.050730 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 805/1620] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:34.055165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5834acb-7b3b-4629-b878-fb3195c33227 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:34.067694 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5834acb-7b3b-4629-b878-fb3195c33227 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:34.075042 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c327bd0-633c-4f75-b7ef-73e5edd79a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:34.075876 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c327bd0-633c-4f75-b7ef-73e5edd79a81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:34.076161 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c327bd0-633c-4f75-b7ef-73e5edd79a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:34.076341 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c327bd0-633c-4f75-b7ef-73e5edd79a81 admin admin] PATH_INFO: `/v3/projects/c7755054241d44adb6c8f302f1e67fe9/users/643e03792f99411b82afc1fe5e2250d5/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:34.078335 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c327bd0-633c-4f75-b7ef-73e5edd79a81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7755054241d44adb6c8f302f1e67fe9, user_id=643e03792f99411b82afc1fe5e2250d5, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:34.081623 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c327bd0-633c-4f75-b7ef-73e5edd79a81 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:34.101290 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 816/1621] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:34 2022] PUT /identity/v3/projects/c7755054241d44adb6c8f302f1e67fe9/users/643e03792f99411b82afc1fe5e2250d5/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:34.106400 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233049b5-29e7-423e-8f2e-f430856e1cce None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:34.106758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233049b5-29e7-423e-8f2e-f430856e1cce None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:34.107103 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233049b5-29e7-423e-8f2e-f430856e1cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:34.142107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-233049b5-29e7-423e-8f2e-f430856e1cce None None] MFA Rules not processed for user `643e03792f99411b82afc1fe5e2250d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:34.144068 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-233049b5-29e7-423e-8f2e-f430856e1cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:34.231938 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074ee20-17db-4e10-ac45-ecf373ccf84d None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:34.232237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074ee20-17db-4e10-ac45-ecf373ccf84d None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:34.232505 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074ee20-17db-4e10-ac45-ecf373ccf84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:34.236582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 806/1622] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:34 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:34.248772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233049b5-29e7-423e-8f2e-f430856e1cce None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:34.250669 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-233049b5-29e7-423e-8f2e-f430856e1cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:34.266602 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-5074ee20-17db-4e10-ac45-ecf373ccf84d None None] MFA Rules not processed for user `4e6fe9efa7f8487b96370ab4f99dc21c`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:34.267821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-5074ee20-17db-4e10-ac45-ecf373ccf84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:34.278949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:34.290262 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:34.329087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233049b5-29e7-423e-8f2e-f430856e1cce None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:34.333235 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56327b1-667d-42e9-a390-e3eaa26f5cbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:34.334038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56327b1-667d-42e9-a390-e3eaa26f5cbb service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:34.334484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56327b1-667d-42e9-a390-e3eaa26f5cbb service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:34.334911 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56327b1-667d-42e9-a390-e3eaa26f5cbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:34.350467 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56327b1-667d-42e9-a390-e3eaa26f5cbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:34.352114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56327b1-667d-42e9-a390-e3eaa26f5cbb service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:34.360906 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 817/1623] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:36:34.371827 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 807/1624] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:34.372807 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074ee20-17db-4e10-ac45-ecf373ccf84d None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:34.374471 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-5074ee20-17db-4e10-ac45-ecf373ccf84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:34.403965 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074ee20-17db-4e10-ac45-ecf373ccf84d None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:34.414914 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d1ae0-1b0e-4d74-b9ad-3e1cad5169f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:34.415508 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d1ae0-1b0e-4d74-b9ad-3e1cad5169f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:34.415792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d1ae0-1b0e-4d74-b9ad-3e1cad5169f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:34.416066 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d1ae0-1b0e-4d74-b9ad-3e1cad5169f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:34.515906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56327b1-667d-42e9-a390-e3eaa26f5cbb service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:34.519703 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494d1ae0-1b0e-4d74-b9ad-3e1cad5169f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:34.521214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494d1ae0-1b0e-4d74-b9ad-3e1cad5169f9 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:34.529516 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56327b1-667d-42e9-a390-e3eaa26f5cbb service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:34.534479 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 818/1625] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:36:34.538936 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d9ecd-4b8f-4df4-8234-b1cefaa2da95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:34.539897 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d9ecd-4b8f-4df4-8234-b1cefaa2da95 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:34.540470 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d9ecd-4b8f-4df4-8234-b1cefaa2da95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:34.541032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d9ecd-4b8f-4df4-8234-b1cefaa2da95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:34.638957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d9ecd-4b8f-4df4-8234-b1cefaa2da95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:34.640498 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d9ecd-4b8f-4df4-8234-b1cefaa2da95 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:34.652120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 808/1626] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:36:35.973988 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d1ae0-1b0e-4d74-b9ad-3e1cad5169f9 service cinder] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:35.982821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d1ae0-1b0e-4d74-b9ad-3e1cad5169f9 service cinder] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:35.986849 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ad02b-2fcb-42f3-b688-f4b42baf705f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:35.988050 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ad02b-2fcb-42f3-b688-f4b42baf705f service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:35.988753 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ad02b-2fcb-42f3-b688-f4b42baf705f service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:35.989155 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ad02b-2fcb-42f3-b688-f4b42baf705f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:35.999362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ad02b-2fcb-42f3-b688-f4b42baf705f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:36.000949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ad02b-2fcb-42f3-b688-f4b42baf705f service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:36.012163 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 819/1627] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:37.719581 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d9ecd-4b8f-4df4-8234-b1cefaa2da95 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:37.898494 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d9ecd-4b8f-4df4-8234-b1cefaa2da95 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:37.914464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380798c1-dfb1-4e0a-b505-7e5c6b415b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:37.915232 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380798c1-dfb1-4e0a-b505-7e5c6b415b17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:37.915632 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380798c1-dfb1-4e0a-b505-7e5c6b415b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:37.916290 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380798c1-dfb1-4e0a-b505-7e5c6b415b17 admin admin] PATH_INFO: `/v3/users/2b2f3f7965e54830a074dbde9031220b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:37.917572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380798c1-dfb1-4e0a-b505-7e5c6b415b17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b2f3f7965e54830a074dbde9031220b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:37.921746 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380798c1-dfb1-4e0a-b505-7e5c6b415b17 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:38.040110 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-380798c1-dfb1-4e0a-b505-7e5c6b415b17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b2f3f7965e54830a074dbde9031220b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:38.041461 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-380798c1-dfb1-4e0a-b505-7e5c6b415b17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b2f3f7965e54830a074dbde9031220b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:38.061585 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-380798c1-dfb1-4e0a-b505-7e5c6b415b17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b2f3f7965e54830a074dbde9031220b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:38.076800 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-380798c1-dfb1-4e0a-b505-7e5c6b415b17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b2f3f7965e54830a074dbde9031220b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:38.077693 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-380798c1-dfb1-4e0a-b505-7e5c6b415b17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b2f3f7965e54830a074dbde9031220b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:38.081951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:38.113644 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 809/1628] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:37 2022] DELETE /identity/v3/users/2b2f3f7965e54830a074dbde9031220b => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:38.115003 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae88676-43a2-4e0a-91b7-d717a0e77290 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:38.115453 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae88676-43a2-4e0a-91b7-d717a0e77290 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:38.115523 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae88676-43a2-4e0a-91b7-d717a0e77290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:38.143732 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-2ae88676-43a2-4e0a-91b7-d717a0e77290 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:38.145195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-2ae88676-43a2-4e0a-91b7-d717a0e77290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:38.258120 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 820/1629] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:38.281328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380798c1-dfb1-4e0a-b505-7e5c6b415b17 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:38.283402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-380798c1-dfb1-4e0a-b505-7e5c6b415b17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:38.444621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380798c1-dfb1-4e0a-b505-7e5c6b415b17 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:38.456568 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4453a5-719a-4f1d-bf6a-0fba60297442 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:38.457238 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4453a5-719a-4f1d-bf6a-0fba60297442 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:38.457524 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4453a5-719a-4f1d-bf6a-0fba60297442 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:38.457794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4453a5-719a-4f1d-bf6a-0fba60297442 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:38.480553 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4453a5-719a-4f1d-bf6a-0fba60297442 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:38.483385 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4453a5-719a-4f1d-bf6a-0fba60297442 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:38.509167 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 810/1630] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:38.524362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae88676-43a2-4e0a-91b7-d717a0e77290 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:38.526526 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-2ae88676-43a2-4e0a-91b7-d717a0e77290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:38.548616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae88676-43a2-4e0a-91b7-d717a0e77290 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:38.566036 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a7f2a5-bc0e-496c-830f-4523f2b5b59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:38.566977 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7f2a5-bc0e-496c-830f-4523f2b5b59b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:38.567399 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7f2a5-bc0e-496c-830f-4523f2b5b59b admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:38.567811 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7f2a5-bc0e-496c-830f-4523f2b5b59b admin admin] PATH_INFO: `/v3/users/dd2e24f606434ede8b6e9d68312d144c` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:38.569373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a7f2a5-bc0e-496c-830f-4523f2b5b59b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd2e24f606434ede8b6e9d68312d144c)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:38.571954 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a7f2a5-bc0e-496c-830f-4523f2b5b59b admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:38.653577 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:38.678116 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-55a7f2a5-bc0e-496c-830f-4523f2b5b59b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd2e24f606434ede8b6e9d68312d144c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:38.679386 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-55a7f2a5-bc0e-496c-830f-4523f2b5b59b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd2e24f606434ede8b6e9d68312d144c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:38.694718 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-55a7f2a5-bc0e-496c-830f-4523f2b5b59b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd2e24f606434ede8b6e9d68312d144c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:38.695629 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-55a7f2a5-bc0e-496c-830f-4523f2b5b59b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd2e24f606434ede8b6e9d68312d144c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:38.707886 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-55a7f2a5-bc0e-496c-830f-4523f2b5b59b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd2e24f606434ede8b6e9d68312d144c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:38.753617 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 821/1631] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:38 2022] DELETE /identity/v3/users/dd2e24f606434ede8b6e9d68312d144c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:38.763566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dea4d6-1364-443d-bbfd-eeeb46d5da3d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:38.764072 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dea4d6-1364-443d-bbfd-eeeb46d5da3d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:38.764230 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dea4d6-1364-443d-bbfd-eeeb46d5da3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:38.813346 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-64dea4d6-1364-443d-bbfd-eeeb46d5da3d None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:38.816294 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-64dea4d6-1364-443d-bbfd-eeeb46d5da3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:38.973055 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 811/1632] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:38.982017 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a7f2a5-bc0e-496c-830f-4523f2b5b59b admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:38.983897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-55a7f2a5-bc0e-496c-830f-4523f2b5b59b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:39.056220 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a7f2a5-bc0e-496c-830f-4523f2b5b59b admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:39.061758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e878313-08bf-40c7-8a56-ee26a056e804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:39.062358 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e878313-08bf-40c7-8a56-ee26a056e804 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:39.062772 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e878313-08bf-40c7-8a56-ee26a056e804 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:39.063235 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e878313-08bf-40c7-8a56-ee26a056e804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:39.078865 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e878313-08bf-40c7-8a56-ee26a056e804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:39.081473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e878313-08bf-40c7-8a56-ee26a056e804 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:39.093093 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dea4d6-1364-443d-bbfd-eeeb46d5da3d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:39.096310 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-64dea4d6-1364-443d-bbfd-eeeb46d5da3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:39.109350 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 822/1633] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:39.113867 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dea4d6-1364-443d-bbfd-eeeb46d5da3d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:39.119838 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7267a090-0baa-4c15-aba9-2561846b207a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:39.120711 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267a090-0baa-4c15-aba9-2561846b207a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:39.121216 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267a090-0baa-4c15-aba9-2561846b207a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:39.121672 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267a090-0baa-4c15-aba9-2561846b207a admin admin] PATH_INFO: `/v3/projects/b272e5b187874e73b3356ec14840027b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:39.123106 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7267a090-0baa-4c15-aba9-2561846b207a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b272e5b187874e73b3356ec14840027b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:39.126055 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7267a090-0baa-4c15-aba9-2561846b207a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:39.195142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7267a090-0baa-4c15-aba9-2561846b207a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b272e5b187874e73b3356ec14840027b', 'request_id': 'req-7267a090-0baa-4c15-aba9-2561846b207a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:39.208836 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7267a090-0baa-4c15-aba9-2561846b207a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b272e5b187874e73b3356ec14840027b', 'request_id': 'req-7267a090-0baa-4c15-aba9-2561846b207a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:39.219908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7267a090-0baa-4c15-aba9-2561846b207a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b272e5b187874e73b3356ec14840027b', 'request_id': 'req-7267a090-0baa-4c15-aba9-2561846b207a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:39.236302 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7267a090-0baa-4c15-aba9-2561846b207a admin admin] The token cache is being invalidate because project b272e5b187874e73b3356ec14840027b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:39.236624 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7267a090-0baa-4c15-aba9-2561846b207a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:39.238610 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7267a090-0baa-4c15-aba9-2561846b207a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:39.246939 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 812/1634] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:39 2022] DELETE /identity/v3/projects/b272e5b187874e73b3356ec14840027b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:39.251017 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e878313-08bf-40c7-8a56-ee26a056e804 service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:39.269229 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e878313-08bf-40c7-8a56-ee26a056e804 service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:39.363897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3279e-75e9-4ce0-977d-8a890c4f0be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:39.364976 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3279e-75e9-4ce0-977d-8a890c4f0be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:39.365435 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3279e-75e9-4ce0-977d-8a890c4f0be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:39.365877 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3279e-75e9-4ce0-977d-8a890c4f0be5 admin admin] PATH_INFO: `/v3/users/643e03792f99411b82afc1fe5e2250d5` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:39.367491 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc3279e-75e9-4ce0-977d-8a890c4f0be5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=643e03792f99411b82afc1fe5e2250d5)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:39.376118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc3279e-75e9-4ce0-977d-8a890c4f0be5 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:39.456910 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-fbc3279e-75e9-4ce0-977d-8a890c4f0be5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '643e03792f99411b82afc1fe5e2250d5'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:39.456910 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-fbc3279e-75e9-4ce0-977d-8a890c4f0be5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '643e03792f99411b82afc1fe5e2250d5'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:39.457720 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-fbc3279e-75e9-4ce0-977d-8a890c4f0be5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '643e03792f99411b82afc1fe5e2250d5'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:39.459807 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-fbc3279e-75e9-4ce0-977d-8a890c4f0be5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '643e03792f99411b82afc1fe5e2250d5'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:39.469473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:39.474638 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-fbc3279e-75e9-4ce0-977d-8a890c4f0be5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '643e03792f99411b82afc1fe5e2250d5'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:39.499734 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 823/1635] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:39 2022] DELETE /identity/v3/users/643e03792f99411b82afc1fe5e2250d5 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:39.757754 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7267a090-0baa-4c15-aba9-2561846b207a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:39.759478 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7267a090-0baa-4c15-aba9-2561846b207a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:39.782018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7267a090-0baa-4c15-aba9-2561846b207a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:39.809067 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:39.862328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3279e-75e9-4ce0-977d-8a890c4f0be5 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:39.864253 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-fbc3279e-75e9-4ce0-977d-8a890c4f0be5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:39.887709 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90bc888-7a3a-4453-ad6e-8548adc1801a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:39.889602 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90bc888-7a3a-4453-ad6e-8548adc1801a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:39.890103 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90bc888-7a3a-4453-ad6e-8548adc1801a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:39.890516 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90bc888-7a3a-4453-ad6e-8548adc1801a admin admin] PATH_INFO: `/v3/projects/db84fe89e0934db69643734c541c9824` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:39.891860 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90bc888-7a3a-4453-ad6e-8548adc1801a admin admin] RBAC: Authorizing `identity:delete_project(project_id=db84fe89e0934db69643734c541c9824)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:39.895144 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90bc888-7a3a-4453-ad6e-8548adc1801a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:39.975007 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3279e-75e9-4ce0-977d-8a890c4f0be5 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:39.981881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc94901-9fc7-49d5-9194-6128a92ac3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:39.982710 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc94901-9fc7-49d5-9194-6128a92ac3ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:39.984373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc94901-9fc7-49d5-9194-6128a92ac3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:39.984808 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc94901-9fc7-49d5-9194-6128a92ac3ff admin admin] PATH_INFO: `/v3/projects/c7755054241d44adb6c8f302f1e67fe9` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:39.986192 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc94901-9fc7-49d5-9194-6128a92ac3ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7755054241d44adb6c8f302f1e67fe9)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:39.989202 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc94901-9fc7-49d5-9194-6128a92ac3ff admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:39.999581 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a90bc888-7a3a-4453-ad6e-8548adc1801a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db84fe89e0934db69643734c541c9824', 'request_id': 'req-a90bc888-7a3a-4453-ad6e-8548adc1801a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:40.008907 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a90bc888-7a3a-4453-ad6e-8548adc1801a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db84fe89e0934db69643734c541c9824', 'request_id': 'req-a90bc888-7a3a-4453-ad6e-8548adc1801a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:40.019605 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a90bc888-7a3a-4453-ad6e-8548adc1801a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db84fe89e0934db69643734c541c9824', 'request_id': 'req-a90bc888-7a3a-4453-ad6e-8548adc1801a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:40.037494 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a90bc888-7a3a-4453-ad6e-8548adc1801a admin admin] The token cache is being invalidate because project db84fe89e0934db69643734c541c9824 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:40.037794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a90bc888-7a3a-4453-ad6e-8548adc1801a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:40.038450 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a90bc888-7a3a-4453-ad6e-8548adc1801a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:40.040390 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 813/1636] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:39 2022] DELETE /identity/v3/projects/db84fe89e0934db69643734c541c9824 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:40.044329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90bc888-7a3a-4453-ad6e-8548adc1801a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:40.046062 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a90bc888-7a3a-4453-ad6e-8548adc1801a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:40.079726 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6dc94901-9fc7-49d5-9194-6128a92ac3ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7755054241d44adb6c8f302f1e67fe9', 'request_id': 'req-6dc94901-9fc7-49d5-9194-6128a92ac3ff'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:40.099821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6dc94901-9fc7-49d5-9194-6128a92ac3ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7755054241d44adb6c8f302f1e67fe9', 'request_id': 'req-6dc94901-9fc7-49d5-9194-6128a92ac3ff'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:40.114839 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6dc94901-9fc7-49d5-9194-6128a92ac3ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7755054241d44adb6c8f302f1e67fe9', 'request_id': 'req-6dc94901-9fc7-49d5-9194-6128a92ac3ff'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:40.133858 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6dc94901-9fc7-49d5-9194-6128a92ac3ff admin admin] The token cache is being invalidate because project c7755054241d44adb6c8f302f1e67fe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:40.134372 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6dc94901-9fc7-49d5-9194-6128a92ac3ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:40.135258 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6dc94901-9fc7-49d5-9194-6128a92ac3ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:40.137284 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 824/1637] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:39 2022] DELETE /identity/v3/projects/c7755054241d44adb6c8f302f1e67fe9 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:36:40.142838 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507cf07-2527-4621-8f50-0f780d06d36f None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:40.143664 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507cf07-2527-4621-8f50-0f780d06d36f None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:40.144104 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507cf07-2527-4621-8f50-0f780d06d36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:40.155920 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90bc888-7a3a-4453-ad6e-8548adc1801a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:40.174798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-6507cf07-2527-4621-8f50-0f780d06d36f None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:40.182041 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-6507cf07-2527-4621-8f50-0f780d06d36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:40.267761 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:40.280553 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 825/1638] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:36:40.282193 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6507cf07-2527-4621-8f50-0f780d06d36f None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:40.284415 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-6507cf07-2527-4621-8f50-0f780d06d36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:40.298165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6507cf07-2527-4621-8f50-0f780d06d36f None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:40.300170 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da191780-152a-4b2b-9b94-29ee5e2140ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:40.300961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da191780-152a-4b2b-9b94-29ee5e2140ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:40.301344 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da191780-152a-4b2b-9b94-29ee5e2140ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:40.301619 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da191780-152a-4b2b-9b94-29ee5e2140ea admin admin] PATH_INFO: `/v3/users/7617b4ac9cc24217865b05b2837a2c1b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:40.303571 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da191780-152a-4b2b-9b94-29ee5e2140ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=7617b4ac9cc24217865b05b2837a2c1b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:40.306223 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da191780-152a-4b2b-9b94-29ee5e2140ea admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:40.311132 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f7ef8-9616-4733-afad-df08f54c067a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:40.311782 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f7ef8-9616-4733-afad-df08f54c067a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:40.312058 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f7ef8-9616-4733-afad-df08f54c067a admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:40.312325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f7ef8-9616-4733-afad-df08f54c067a admin admin] PATH_INFO: `/v3/users/933723f51b144c5081c8b280e3251d9f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:40.313667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f7ef8-9616-4733-afad-df08f54c067a admin admin] RBAC: Authorizing `identity:delete_user(user_id=933723f51b144c5081c8b280e3251d9f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:40.317653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f7ef8-9616-4733-afad-df08f54c067a admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:40.319476 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:40.416509 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-da191780-152a-4b2b-9b94-29ee5e2140ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7617b4ac9cc24217865b05b2837a2c1b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:40.418325 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-da191780-152a-4b2b-9b94-29ee5e2140ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7617b4ac9cc24217865b05b2837a2c1b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:40.433280 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-da191780-152a-4b2b-9b94-29ee5e2140ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7617b4ac9cc24217865b05b2837a2c1b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:40.441863 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-862f7ef8-9616-4733-afad-df08f54c067a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '933723f51b144c5081c8b280e3251d9f'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:40.443102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-862f7ef8-9616-4733-afad-df08f54c067a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '933723f51b144c5081c8b280e3251d9f'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:40.448337 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-da191780-152a-4b2b-9b94-29ee5e2140ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7617b4ac9cc24217865b05b2837a2c1b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:40.450416 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-da191780-152a-4b2b-9b94-29ee5e2140ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7617b4ac9cc24217865b05b2837a2c1b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:40.461691 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-862f7ef8-9616-4733-afad-df08f54c067a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '933723f51b144c5081c8b280e3251d9f'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:40.464665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-862f7ef8-9616-4733-afad-df08f54c067a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '933723f51b144c5081c8b280e3251d9f'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:40.478508 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 814/1639] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:40 2022] DELETE /identity/v3/users/7617b4ac9cc24217865b05b2837a2c1b => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:40.478676 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-862f7ef8-9616-4733-afad-df08f54c067a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '933723f51b144c5081c8b280e3251d9f'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:40.478761 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c571564-b68b-4398-b3cc-d1f51d7df8ce None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:40.478761 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c571564-b68b-4398-b3cc-d1f51d7df8ce None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:40.478761 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c571564-b68b-4398-b3cc-d1f51d7df8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:40.511446 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2c571564-b68b-4398-b3cc-d1f51d7df8ce None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:40.512787 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2c571564-b68b-4398-b3cc-d1f51d7df8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:40.606456 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 826/1640] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:40 2022] DELETE /identity/v3/users/933723f51b144c5081c8b280e3251d9f => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:40.610956 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9021c09-f4fa-47b3-845b-0f25ec10842a None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:40.611236 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9021c09-f4fa-47b3-845b-0f25ec10842a None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:40.611562 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9021c09-f4fa-47b3-845b-0f25ec10842a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:40.637801 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-d9021c09-f4fa-47b3-845b-0f25ec10842a None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:40.638897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-d9021c09-f4fa-47b3-845b-0f25ec10842a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:40.641189 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 815/1641] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:40.644586 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c571564-b68b-4398-b3cc-d1f51d7df8ce None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:40.682452 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c571564-b68b-4398-b3cc-d1f51d7df8ce None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:40.787728 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 827/1642] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:40.798573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9021c09-f4fa-47b3-845b-0f25ec10842a None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:40.800832 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f586d3c-b985-4d7b-a0d2-a6e7ce676e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:40.801488 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f586d3c-b985-4d7b-a0d2-a6e7ce676e57 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:40.801768 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f586d3c-b985-4d7b-a0d2-a6e7ce676e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:40.802036 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f586d3c-b985-4d7b-a0d2-a6e7ce676e57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:40.802930 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f586d3c-b985-4d7b-a0d2-a6e7ce676e57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:36:40.803051 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f586d3c-b985-4d7b-a0d2-a6e7ce676e57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:40.803838 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-d9021c09-f4fa-47b3-845b-0f25ec10842a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:40.804813 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f586d3c-b985-4d7b-a0d2-a6e7ce676e57 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:40.813937 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:40.820797 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 816/1643] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:36:40.822901 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f586d3c-b985-4d7b-a0d2-a6e7ce676e57 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:40.833877 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f586d3c-b985-4d7b-a0d2-a6e7ce676e57 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:40.839294 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52497731-7997-4e94-91a3-181adc4aeb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:40.839881 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52497731-7997-4e94-91a3-181adc4aeb7d admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:40.840346 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52497731-7997-4e94-91a3-181adc4aeb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:40.840464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52497731-7997-4e94-91a3-181adc4aeb7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:40.841138 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52497731-7997-4e94-91a3-181adc4aeb7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:40.843141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52497731-7997-4e94-91a3-181adc4aeb7d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:40.889041 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 817/1644] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:40 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:40.897133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52497731-7997-4e94-91a3-181adc4aeb7d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:40.919048 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9021c09-f4fa-47b3-845b-0f25ec10842a None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:41.014346 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:41.041451 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52497731-7997-4e94-91a3-181adc4aeb7d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:41.046745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6c72d-e72d-43ec-9821-2fc7b63d9472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:41.049993 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6c72d-e72d-43ec-9821-2fc7b63d9472 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:41.050433 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6c72d-e72d-43ec-9821-2fc7b63d9472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:41.050718 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6c72d-e72d-43ec-9821-2fc7b63d9472 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:41.051394 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6c72d-e72d-43ec-9821-2fc7b63d9472 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:41.052792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f61459-f7ef-45cb-ae4e-5bb62e57b53e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:41.053396 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6c72d-e72d-43ec-9821-2fc7b63d9472 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:41.054654 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f61459-f7ef-45cb-ae4e-5bb62e57b53e service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:41.055583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f61459-f7ef-45cb-ae4e-5bb62e57b53e service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:41.056379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f61459-f7ef-45cb-ae4e-5bb62e57b53e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:41.070051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f61459-f7ef-45cb-ae4e-5bb62e57b53e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:41.071785 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f61459-f7ef-45cb-ae4e-5bb62e57b53e service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:41.083558 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 828/1645] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:41.084834 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f61459-f7ef-45cb-ae4e-5bb62e57b53e service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:41.085805 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 818/1646] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:40 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:41.086635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-e6f61459-f7ef-45cb-ae4e-5bb62e57b53e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:41.089331 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6c72d-e72d-43ec-9821-2fc7b63d9472 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:41.119557 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6c72d-e72d-43ec-9821-2fc7b63d9472 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:41.120346 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f61459-f7ef-45cb-ae4e-5bb62e57b53e service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:41.132216 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d75859-261b-4a17-8592-12221be77563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:41.133441 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62e90c-604c-4992-bc7b-aa6ed4e893ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:41.134097 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62e90c-604c-4992-bc7b-aa6ed4e893ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:41.134376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62e90c-604c-4992-bc7b-aa6ed4e893ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:41.134651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62e90c-604c-4992-bc7b-aa6ed4e893ad admin admin] PATH_INFO: `/v3/projects/5ac3e8306f2946b3aaa0978bf6337e04` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:41.135703 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d75859-261b-4a17-8592-12221be77563 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:41.135981 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d75859-261b-4a17-8592-12221be77563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:41.136252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d75859-261b-4a17-8592-12221be77563 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:41.136946 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d75859-261b-4a17-8592-12221be77563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:41.138885 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62e90c-604c-4992-bc7b-aa6ed4e893ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ac3e8306f2946b3aaa0978bf6337e04)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:41.140961 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62e90c-604c-4992-bc7b-aa6ed4e893ad admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:41.143489 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d75859-261b-4a17-8592-12221be77563 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:41.166608 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 819/1647] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:41.170964 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d75859-261b-4a17-8592-12221be77563 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:41.196855 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d75859-261b-4a17-8592-12221be77563 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:41.208847 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a75de-8a4b-48c9-90ab-1b18cb34e2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:41.209532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a75de-8a4b-48c9-90ab-1b18cb34e2fc admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:41.210112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a75de-8a4b-48c9-90ab-1b18cb34e2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:41.210326 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a75de-8a4b-48c9-90ab-1b18cb34e2fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:41.211044 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a75de-8a4b-48c9-90ab-1b18cb34e2fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:41.220694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a75de-8a4b-48c9-90ab-1b18cb34e2fc admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:41.245812 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 820/1648] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:41.251817 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a75de-8a4b-48c9-90ab-1b18cb34e2fc admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:41.253436 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-db62e90c-604c-4992-bc7b-aa6ed4e893ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ac3e8306f2946b3aaa0978bf6337e04', 'request_id': 'req-db62e90c-604c-4992-bc7b-aa6ed4e893ad'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:41.271245 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-db62e90c-604c-4992-bc7b-aa6ed4e893ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ac3e8306f2946b3aaa0978bf6337e04', 'request_id': 'req-db62e90c-604c-4992-bc7b-aa6ed4e893ad'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:41.272790 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a75de-8a4b-48c9-90ab-1b18cb34e2fc admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:41.290362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-db62e90c-604c-4992-bc7b-aa6ed4e893ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ac3e8306f2946b3aaa0978bf6337e04', 'request_id': 'req-db62e90c-604c-4992-bc7b-aa6ed4e893ad'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:41.312013 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-db62e90c-604c-4992-bc7b-aa6ed4e893ad admin admin] The token cache is being invalidate because project 5ac3e8306f2946b3aaa0978bf6337e04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:41.312259 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-db62e90c-604c-4992-bc7b-aa6ed4e893ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:41.314059 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-db62e90c-604c-4992-bc7b-aa6ed4e893ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:41.320182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 829/1649] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:41 2022] DELETE /identity/v3/projects/5ac3e8306f2946b3aaa0978bf6337e04 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:41.323202 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367be49-551e-4868-9292-afb7dbbec199 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:41.323682 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367be49-551e-4868-9292-afb7dbbec199 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:41.324198 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367be49-551e-4868-9292-afb7dbbec199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:41.352515 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-0367be49-551e-4868-9292-afb7dbbec199 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:41.353972 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-0367be49-551e-4868-9292-afb7dbbec199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:41.406096 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e5921-87f9-4931-af6a-0967bb416e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:41.406781 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e5921-87f9-4931-af6a-0967bb416e73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:41.407066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e5921-87f9-4931-af6a-0967bb416e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:41.407350 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e5921-87f9-4931-af6a-0967bb416e73 admin admin] PATH_INFO: `/v3/projects/0b6a8574a0d34818811deb8d18a7011b/users/aafdb707212744ca92e317cda05044db/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:41.448225 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e5921-87f9-4931-af6a-0967bb416e73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b6a8574a0d34818811deb8d18a7011b, user_id=aafdb707212744ca92e317cda05044db, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:41.452162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 830/1650] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:41.458316 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e5921-87f9-4931-af6a-0967bb416e73 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:41.461575 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c08f6-0f8f-4bda-bc81-a174b790cbbd None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:41.461968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c08f6-0f8f-4bda-bc81-a174b790cbbd None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:41.462492 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c08f6-0f8f-4bda-bc81-a174b790cbbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:41.487439 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 821/1651] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:41 2022] PUT /identity/v3/projects/0b6a8574a0d34818811deb8d18a7011b/users/aafdb707212744ca92e317cda05044db/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:36:41.498008 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58f81d-b315-4cca-97f1-3411590f17ad None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:41.498291 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58f81d-b315-4cca-97f1-3411590f17ad None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:41.498623 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58f81d-b315-4cca-97f1-3411590f17ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:41.519681 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-de0c08f6-0f8f-4bda-bc81-a174b790cbbd None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:41.520986 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-de0c08f6-0f8f-4bda-bc81-a174b790cbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:41.575992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-4a58f81d-b315-4cca-97f1-3411590f17ad None None] MFA Rules not processed for user `aafdb707212744ca92e317cda05044db`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:41.577294 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4a58f81d-b315-4cca-97f1-3411590f17ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:41.599389 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:41.637249 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 831/1652] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:41.642051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c08f6-0f8f-4bda-bc81-a174b790cbbd None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:41.648727 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:41.656106 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c08f6-0f8f-4bda-bc81-a174b790cbbd None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:41.666310 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133c0a7-c464-4a4f-bec9-c46ebc41303c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:41.667176 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133c0a7-c464-4a4f-bec9-c46ebc41303c admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:41.667817 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133c0a7-c464-4a4f-bec9-c46ebc41303c admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:41.668383 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133c0a7-c464-4a4f-bec9-c46ebc41303c admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:41.669303 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133c0a7-c464-4a4f-bec9-c46ebc41303c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:36:41.669688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133c0a7-c464-4a4f-bec9-c46ebc41303c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:41.671515 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133c0a7-c464-4a4f-bec9-c46ebc41303c admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:41.691242 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 832/1653] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:36:41.692501 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133c0a7-c464-4a4f-bec9-c46ebc41303c admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:41.713028 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 822/1654] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:41.716211 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58f81d-b315-4cca-97f1-3411590f17ad None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:41.716524 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133c0a7-c464-4a4f-bec9-c46ebc41303c admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:41.717883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4a58f81d-b315-4cca-97f1-3411590f17ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:41.726013 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c36808-38be-4b62-bb86-933246efb943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:41.726826 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c36808-38be-4b62-bb86-933246efb943 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:41.727230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c36808-38be-4b62-bb86-933246efb943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:41.727538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c36808-38be-4b62-bb86-933246efb943 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:41.728681 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c36808-38be-4b62-bb86-933246efb943 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:41.730504 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c36808-38be-4b62-bb86-933246efb943 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:41.733116 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58f81d-b315-4cca-97f1-3411590f17ad None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:41.739416 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6ff8f8-1c34-40a4-9dc7-97c82ea2a4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:41.740255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ff8f8-1c34-40a4-9dc7-97c82ea2a4ba admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:41.740545 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ff8f8-1c34-40a4-9dc7-97c82ea2a4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:41.741381 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ff8f8-1c34-40a4-9dc7-97c82ea2a4ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:41.743066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6ff8f8-1c34-40a4-9dc7-97c82ea2a4ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:41.745039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6ff8f8-1c34-40a4-9dc7-97c82ea2a4ba admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:41.783207 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 833/1655] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:41 2022] POST /identity/v3/projects => generated 365 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:41.785110 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c36808-38be-4b62-bb86-933246efb943 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:41.787471 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-d8c36808-38be-4b62-bb86-933246efb943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:41.808199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 823/1656] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:41 2022] POST /identity/v3/projects => generated 383 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:36:41.812674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6ff8f8-1c34-40a4-9dc7-97c82ea2a4ba admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:41.868766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c36808-38be-4b62-bb86-933246efb943 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:41.901257 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6ff8f8-1c34-40a4-9dc7-97c82ea2a4ba admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:41.908309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ef9e8-c227-4051-af8a-d4bc790272bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:41.909791 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ef9e8-c227-4051-af8a-d4bc790272bd admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:41.910076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ef9e8-c227-4051-af8a-d4bc790272bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:41.910351 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ef9e8-c227-4051-af8a-d4bc790272bd admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:41.910998 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ef9e8-c227-4051-af8a-d4bc790272bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:41.913074 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ef9e8-c227-4051-af8a-d4bc790272bd admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:41.943489 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 824/1657] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:41 2022] POST /identity/v3/users => generated 283 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:41.943685 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022851d8-2232-43e0-b015-2716d8e43a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:41.943795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ef9e8-c227-4051-af8a-d4bc790272bd admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:41.943913 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022851d8-2232-43e0-b015-2716d8e43a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:41.943913 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022851d8-2232-43e0-b015-2716d8e43a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:41.943913 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022851d8-2232-43e0-b015-2716d8e43a50 admin admin] PATH_INFO: `/v3/projects/fe5c694c6b584821b95eb97d68a83af1` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:41.943913 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022851d8-2232-43e0-b015-2716d8e43a50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe5c694c6b584821b95eb97d68a83af1)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:41.946495 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022851d8-2232-43e0-b015-2716d8e43a50 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:41.954035 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ef9e8-c227-4051-af8a-d4bc790272bd admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:41.959978 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3bbc2c-575d-48b3-9884-2d83e3e1862d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:41.960911 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3bbc2c-575d-48b3-9884-2d83e3e1862d admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:41.961031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3bbc2c-575d-48b3-9884-2d83e3e1862d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:41.961508 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3bbc2c-575d-48b3-9884-2d83e3e1862d admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:41.962271 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3bbc2c-575d-48b3-9884-2d83e3e1862d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:41.964325 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3bbc2c-575d-48b3-9884-2d83e3e1862d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:41.995397 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 825/1658] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:41 2022] POST /identity/v3/users => generated 298 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:41.995397 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3bbc2c-575d-48b3-9884-2d83e3e1862d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:42.008156 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3bbc2c-575d-48b3-9884-2d83e3e1862d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:42.010561 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-022851d8-2232-43e0-b015-2716d8e43a50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe5c694c6b584821b95eb97d68a83af1', 'request_id': 'req-022851d8-2232-43e0-b015-2716d8e43a50'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:42.019936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-022851d8-2232-43e0-b015-2716d8e43a50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe5c694c6b584821b95eb97d68a83af1', 'request_id': 'req-022851d8-2232-43e0-b015-2716d8e43a50'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:42.043582 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-022851d8-2232-43e0-b015-2716d8e43a50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe5c694c6b584821b95eb97d68a83af1', 'request_id': 'req-022851d8-2232-43e0-b015-2716d8e43a50'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:42.065582 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-022851d8-2232-43e0-b015-2716d8e43a50 admin admin] The token cache is being invalidate because project fe5c694c6b584821b95eb97d68a83af1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:42.065582 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-022851d8-2232-43e0-b015-2716d8e43a50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:42.067750 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-022851d8-2232-43e0-b015-2716d8e43a50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:42.069538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 834/1659] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:41 2022] DELETE /identity/v3/projects/fe5c694c6b584821b95eb97d68a83af1 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:42.070375 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022851d8-2232-43e0-b015-2716d8e43a50 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:42.072575 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-022851d8-2232-43e0-b015-2716d8e43a50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:42.079908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:42.089030 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b6ac2-cb29-4705-919d-c1f0a3faaf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:42.090428 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b6ac2-cb29-4705-919d-c1f0a3faaf53 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:42.091317 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b6ac2-cb29-4705-919d-c1f0a3faaf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:42.091821 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b6ac2-cb29-4705-919d-c1f0a3faaf53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:42.092894 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b6ac2-cb29-4705-919d-c1f0a3faaf53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:42.094899 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b6ac2-cb29-4705-919d-c1f0a3faaf53 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:42.111433 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 826/1660] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:42.113706 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac44a3c-5207-47dd-81c7-8057f936a9b1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:42.114198 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac44a3c-5207-47dd-81c7-8057f936a9b1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:42.114692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac44a3c-5207-47dd-81c7-8057f936a9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:42.124157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:42.137758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9ac44a3c-5207-47dd-81c7-8057f936a9b1 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:42.139202 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9ac44a3c-5207-47dd-81c7-8057f936a9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:42.158023 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 827/1661] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:42.158887 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac44a3c-5207-47dd-81c7-8057f936a9b1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:42.160393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9ac44a3c-5207-47dd-81c7-8057f936a9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:42.172479 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac44a3c-5207-47dd-81c7-8057f936a9b1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:42.178348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26fdf3-c35f-48c1-849b-ca107c8de2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:42.179071 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26fdf3-c35f-48c1-849b-ca107c8de2a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:42.179449 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26fdf3-c35f-48c1-849b-ca107c8de2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:42.179846 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26fdf3-c35f-48c1-849b-ca107c8de2a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:42.180583 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d26fdf3-c35f-48c1-849b-ca107c8de2a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:42.181954 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022851d8-2232-43e0-b015-2716d8e43a50 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:42.182486 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d26fdf3-c35f-48c1-849b-ca107c8de2a9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:42.187604 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be558b0-5b6b-449b-93e7-2abaaee536e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:42.188447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be558b0-5b6b-449b-93e7-2abaaee536e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:42.188895 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be558b0-5b6b-449b-93e7-2abaaee536e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:42.189315 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be558b0-5b6b-449b-93e7-2abaaee536e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:42.191703 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be558b0-5b6b-449b-93e7-2abaaee536e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:42.193754 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be558b0-5b6b-449b-93e7-2abaaee536e5 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:42.203880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 828/1662] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:42.204363 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 835/1663] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:42.206841 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c11f2-18f5-42ba-a9bb-4a78de9263d0 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:42.207544 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c11f2-18f5-42ba-a9bb-4a78de9263d0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:42.208087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c11f2-18f5-42ba-a9bb-4a78de9263d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:42.209088 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be558b0-5b6b-449b-93e7-2abaaee536e5 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:42.228902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be558b0-5b6b-449b-93e7-2abaaee536e5 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:42.234212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae3aee-ade0-40f5-832a-d02221b9eb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:42.234437 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-ca7c11f2-18f5-42ba-a9bb-4a78de9263d0 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:42.234880 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae3aee-ade0-40f5-832a-d02221b9eb90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:42.235158 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae3aee-ade0-40f5-832a-d02221b9eb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:42.235431 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae3aee-ade0-40f5-832a-d02221b9eb90 admin admin] PATH_INFO: `/v3/projects/190e4a2e98cd41f89df5e698bd45bbe4/users/ad55bfa477224aa38bd067ab12ec2c81/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:42.237564 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ca7c11f2-18f5-42ba-a9bb-4a78de9263d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:42.250288 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 829/1664] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:42.251306 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c11f2-18f5-42ba-a9bb-4a78de9263d0 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:42.257417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae3aee-ade0-40f5-832a-d02221b9eb90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=190e4a2e98cd41f89df5e698bd45bbe4, user_id=ad55bfa477224aa38bd067ab12ec2c81, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:42.260973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae3aee-ade0-40f5-832a-d02221b9eb90 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:42.264750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c11f2-18f5-42ba-a9bb-4a78de9263d0 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:42.271750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad274370-9618-4ab4-9dcc-fb685f40cfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:42.272478 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad274370-9618-4ab4-9dcc-fb685f40cfa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:42.272758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad274370-9618-4ab4-9dcc-fb685f40cfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:42.273310 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad274370-9618-4ab4-9dcc-fb685f40cfa2 admin admin] PATH_INFO: `/v3/projects/c18f6dbe6aac4df5aab1fbd1e2837b36/users/3e8609e8b4da4cf28bf82792df5e5f2b/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:42.291163 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 836/1665] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:42 2022] PUT /identity/v3/projects/190e4a2e98cd41f89df5e698bd45bbe4/users/ad55bfa477224aa38bd067ab12ec2c81/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:42.300682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad274370-9618-4ab4-9dcc-fb685f40cfa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c18f6dbe6aac4df5aab1fbd1e2837b36, user_id=3e8609e8b4da4cf28bf82792df5e5f2b, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:42.304900 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad274370-9618-4ab4-9dcc-fb685f40cfa2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:42.307625 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae3aee-ade0-40f5-832a-d02221b9eb90 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:42.327134 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 830/1666] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:42 2022] PUT /identity/v3/projects/c18f6dbe6aac4df5aab1fbd1e2837b36/users/3e8609e8b4da4cf28bf82792df5e5f2b/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:42.327134 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f065a9-aa88-4045-bc89-bbcba1eb5d12 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:42.327134 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f065a9-aa88-4045-bc89-bbcba1eb5d12 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:42.327134 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f065a9-aa88-4045-bc89-bbcba1eb5d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:42.360996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-40f065a9-aa88-4045-bc89-bbcba1eb5d12 None None] MFA Rules not processed for user `ad55bfa477224aa38bd067ab12ec2c81`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:42.363490 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-40f065a9-aa88-4045-bc89-bbcba1eb5d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:42.390755 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae3aee-ade0-40f5-832a-d02221b9eb90 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:42.439635 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 831/1667] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:42.441154 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f065a9-aa88-4045-bc89-bbcba1eb5d12 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:42.470760 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4a119-97a8-4e85-be29-601b5de88892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:42.471341 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4a119-97a8-4e85-be29-601b5de88892 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:42.471619 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4a119-97a8-4e85-be29-601b5de88892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:42.471893 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4a119-97a8-4e85-be29-601b5de88892 admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:42.472650 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4a119-97a8-4e85-be29-601b5de88892 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:36:42.473012 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4a119-97a8-4e85-be29-601b5de88892 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:42.474720 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4a119-97a8-4e85-be29-601b5de88892 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:42.488469 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 837/1668] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:36:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:36:42.489856 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4a119-97a8-4e85-be29-601b5de88892 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:42.514610 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f065a9-aa88-4045-bc89-bbcba1eb5d12 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:42.517588 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4a119-97a8-4e85-be29-601b5de88892 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:42.521495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98187b66-600a-4eed-a31a-afe308f18a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:42.522574 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98187b66-600a-4eed-a31a-afe308f18a97 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:42.522860 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98187b66-600a-4eed-a31a-afe308f18a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:42.523132 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98187b66-600a-4eed-a31a-afe308f18a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:42.523849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98187b66-600a-4eed-a31a-afe308f18a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:42.524751 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877abac-d417-46d2-ac3f-62d589270890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:42.525550 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877abac-d417-46d2-ac3f-62d589270890 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:42.525765 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98187b66-600a-4eed-a31a-afe308f18a97 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:42.526137 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877abac-d417-46d2-ac3f-62d589270890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:42.526464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877abac-d417-46d2-ac3f-62d589270890 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:42.527384 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1877abac-d417-46d2-ac3f-62d589270890 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:42.529836 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1877abac-d417-46d2-ac3f-62d589270890 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:42.539598 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 832/1669] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:42.542062 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98187b66-600a-4eed-a31a-afe308f18a97 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:42.548909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:42.554648 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98187b66-600a-4eed-a31a-afe308f18a97 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:42.560787 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065643d2-676a-4c78-a36a-40c16258839f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:42.561592 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065643d2-676a-4c78-a36a-40c16258839f admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:42.561947 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065643d2-676a-4c78-a36a-40c16258839f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:42.562220 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065643d2-676a-4c78-a36a-40c16258839f admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:42.562912 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065643d2-676a-4c78-a36a-40c16258839f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:42.565589 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065643d2-676a-4c78-a36a-40c16258839f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:42.571737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 838/1670] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:42 2022] POST /identity/v3/projects => generated 363 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:42.572752 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877abac-d417-46d2-ac3f-62d589270890 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:42.597150 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 833/1671] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:42 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:42.597721 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065643d2-676a-4c78-a36a-40c16258839f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:42.626717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:42.652855 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877abac-d417-46d2-ac3f-62d589270890 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:42.678909 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065643d2-676a-4c78-a36a-40c16258839f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:42.685185 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa16d6-8056-4456-ab4b-bd9794c5e9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:42.685944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa16d6-8056-4456-ab4b-bd9794c5e9cb admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:42.686323 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa16d6-8056-4456-ab4b-bd9794c5e9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:42.686699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa16d6-8056-4456-ab4b-bd9794c5e9cb admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:42.687416 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa16d6-8056-4456-ab4b-bd9794c5e9cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:42.690104 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa16d6-8056-4456-ab4b-bd9794c5e9cb admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:42.718198 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 834/1672] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:42 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:42.719512 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa16d6-8056-4456-ab4b-bd9794c5e9cb admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:42.722699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e8247-a7a6-4203-bccd-c3788c16a5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:42.723326 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e8247-a7a6-4203-bccd-c3788c16a5aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:42.723665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e8247-a7a6-4203-bccd-c3788c16a5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:42.723953 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e8247-a7a6-4203-bccd-c3788c16a5aa admin admin] PATH_INFO: `/v3/projects/c18f6dbe6aac4df5aab1fbd1e2837b36/users/3e8609e8b4da4cf28bf82792df5e5f2b/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:42.726309 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e8247-a7a6-4203-bccd-c3788c16a5aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c18f6dbe6aac4df5aab1fbd1e2837b36, user_id=3e8609e8b4da4cf28bf82792df5e5f2b, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:42.729433 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e8247-a7a6-4203-bccd-c3788c16a5aa admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:42.737741 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa16d6-8056-4456-ab4b-bd9794c5e9cb admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:42.741615 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 839/1673] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:42 2022] PUT /identity/v3/projects/c18f6dbe6aac4df5aab1fbd1e2837b36/users/3e8609e8b4da4cf28bf82792df5e5f2b/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:42.743212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e8247-a7a6-4203-bccd-c3788c16a5aa admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:42.798566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ee780-30a7-464a-9298-0dc46cde02be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:42.799777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ee780-30a7-464a-9298-0dc46cde02be admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:42.800061 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ee780-30a7-464a-9298-0dc46cde02be admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:42.800337 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ee780-30a7-464a-9298-0dc46cde02be admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:42.801005 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ee780-30a7-464a-9298-0dc46cde02be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:42.802929 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ee780-30a7-464a-9298-0dc46cde02be admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:42.829134 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e8247-a7a6-4203-bccd-c3788c16a5aa admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:42.833231 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 835/1674] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:42 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:42.834958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e0a9b-fb93-4632-9bf5-a03551f117fb None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:42.835245 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e0a9b-fb93-4632-9bf5-a03551f117fb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:42.835518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e0a9b-fb93-4632-9bf5-a03551f117fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:42.838127 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f5ead-595f-4ad7-b84d-6a1e022261e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:42.838875 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f5ead-595f-4ad7-b84d-6a1e022261e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:42.839225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f5ead-595f-4ad7-b84d-6a1e022261e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:42.839563 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364f5ead-595f-4ad7-b84d-6a1e022261e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:42.840283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f5ead-595f-4ad7-b84d-6a1e022261e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:42.841993 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364f5ead-595f-4ad7-b84d-6a1e022261e5 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:42.857582 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 840/1675] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:42.858302 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f5ead-595f-4ad7-b84d-6a1e022261e5 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:42.875361 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364f5ead-595f-4ad7-b84d-6a1e022261e5 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:42.878062 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b18e0a9b-fb93-4632-9bf5-a03551f117fb None None] MFA Rules not processed for user `3e8609e8b4da4cf28bf82792df5e5f2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:42.879997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b18e0a9b-fb93-4632-9bf5-a03551f117fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:42.881187 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2966ef-020b-4433-8b0c-a71adbf9e49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:42.881798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2966ef-020b-4433-8b0c-a71adbf9e49f admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:42.882075 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2966ef-020b-4433-8b0c-a71adbf9e49f admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:42.882403 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2966ef-020b-4433-8b0c-a71adbf9e49f admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:42.882985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2966ef-020b-4433-8b0c-a71adbf9e49f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:42.884759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2966ef-020b-4433-8b0c-a71adbf9e49f admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:42.898378 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 841/1676] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:42.899454 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2966ef-020b-4433-8b0c-a71adbf9e49f admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:42.910376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2966ef-020b-4433-8b0c-a71adbf9e49f admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:42.920919 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fbe61-7ad4-41ba-8609-98a01ad2270e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:42.921581 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fbe61-7ad4-41ba-8609-98a01ad2270e admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:42.921855 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fbe61-7ad4-41ba-8609-98a01ad2270e admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:42.922174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5fbe61-7ad4-41ba-8609-98a01ad2270e admin admin] PATH_INFO: `/v3/projects/51f1a35e146e4e48a2c10bf15fd2070b/users/1aa5b84710304b9e8cdab672ccb5553b/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:42.939593 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fbe61-7ad4-41ba-8609-98a01ad2270e admin admin] RBAC: Authorizing `identity:create_grant(project_id=51f1a35e146e4e48a2c10bf15fd2070b, user_id=1aa5b84710304b9e8cdab672ccb5553b, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:42.942626 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5fbe61-7ad4-41ba-8609-98a01ad2270e admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:42.952541 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:42.962753 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 842/1677] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:42 2022] PUT /identity/v3/projects/51f1a35e146e4e48a2c10bf15fd2070b/users/1aa5b84710304b9e8cdab672ccb5553b/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:42.962753 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fbe61-7ad4-41ba-8609-98a01ad2270e admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:42.983201 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 836/1678] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:42.983951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e0a9b-fb93-4632-9bf5-a03551f117fb None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:42.995438 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e0a9b-fb93-4632-9bf5-a03551f117fb None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:43.020649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:43.039731 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5fbe61-7ad4-41ba-8609-98a01ad2270e admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:43.046041 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20281253-7a9f-4f06-9ba1-b646b6c62be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:43.047126 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20281253-7a9f-4f06-9ba1-b646b6c62be5 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:43.047639 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20281253-7a9f-4f06-9ba1-b646b6c62be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:43.049570 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20281253-7a9f-4f06-9ba1-b646b6c62be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:43.050448 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20281253-7a9f-4f06-9ba1-b646b6c62be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:43.053147 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20281253-7a9f-4f06-9ba1-b646b6c62be5 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:43.070062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 843/1679] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:43.070062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20281253-7a9f-4f06-9ba1-b646b6c62be5 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:43.071872 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-20281253-7a9f-4f06-9ba1-b646b6c62be5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:43.084572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9a0b8-3a08-49ce-a690-688561b94a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:43.085762 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9a0b8-3a08-49ce-a690-688561b94a13 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:43.086052 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9a0b8-3a08-49ce-a690-688561b94a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:43.086326 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9a0b8-3a08-49ce-a690-688561b94a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:43.087059 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be9a0b8-3a08-49ce-a690-688561b94a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:43.088662 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be9a0b8-3a08-49ce-a690-688561b94a13 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:43.105144 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 837/1680] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:43.106199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9a0b8-3a08-49ce-a690-688561b94a13 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:43.118226 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be9a0b8-3a08-49ce-a690-688561b94a13 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:43.124113 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458e9c92-d57b-408b-b46d-a1947733e2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:43.124900 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e9c92-d57b-408b-b46d-a1947733e2a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:43.125177 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e9c92-d57b-408b-b46d-a1947733e2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:43.125457 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e9c92-d57b-408b-b46d-a1947733e2a7 admin admin] PATH_INFO: `/v3/projects/1b9b8008f9a74bf19c15360e9bf26bb9/users/7d552a2f6d5649afb1a363a8b6fd611d/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:43.148241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20281253-7a9f-4f06-9ba1-b646b6c62be5 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:43.150588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458e9c92-d57b-408b-b46d-a1947733e2a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b9b8008f9a74bf19c15360e9bf26bb9, user_id=7d552a2f6d5649afb1a363a8b6fd611d, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:43.152011 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ec73b-7d52-4114-93b7-adcc32c90242 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:43.152702 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ec73b-7d52-4114-93b7-adcc32c90242 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:43.153107 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ec73b-7d52-4114-93b7-adcc32c90242 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:43.153374 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ec73b-7d52-4114-93b7-adcc32c90242 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:43.153508 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458e9c92-d57b-408b-b46d-a1947733e2a7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:43.155783 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-1f7ec73b-7d52-4114-93b7-adcc32c90242 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:43.172708 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 838/1681] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:43 2022] PUT /identity/v3/projects/1b9b8008f9a74bf19c15360e9bf26bb9/users/7d552a2f6d5649afb1a363a8b6fd611d/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:43.182387 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458e9c92-d57b-408b-b46d-a1947733e2a7 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:43.244028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ec73b-7d52-4114-93b7-adcc32c90242 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:43.246217 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ec73b-7d52-4114-93b7-adcc32c90242 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:43.262091 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 844/1682] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:43.264108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db6878-63b8-4c74-a261-768dc85a900c None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:43.264568 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db6878-63b8-4c74-a261-768dc85a900c None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:43.265010 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db6878-63b8-4c74-a261-768dc85a900c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:43.296607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458e9c92-d57b-408b-b46d-a1947733e2a7 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:43.300235 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-01db6878-63b8-4c74-a261-768dc85a900c None None] MFA Rules not processed for user `7d552a2f6d5649afb1a363a8b6fd611d`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:43.302400 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-01db6878-63b8-4c74-a261-768dc85a900c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:43.304576 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980f4d4-b13d-474b-8fed-2c348d2ec8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:43.305300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980f4d4-b13d-474b-8fed-2c348d2ec8a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:43.305585 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980f4d4-b13d-474b-8fed-2c348d2ec8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:43.305926 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980f4d4-b13d-474b-8fed-2c348d2ec8a5 admin admin] PATH_INFO: `/v3/projects/51f1a35e146e4e48a2c10bf15fd2070b/users/1aa5b84710304b9e8cdab672ccb5553b/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:43.307972 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4980f4d4-b13d-474b-8fed-2c348d2ec8a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51f1a35e146e4e48a2c10bf15fd2070b, user_id=1aa5b84710304b9e8cdab672ccb5553b, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:43.310766 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4980f4d4-b13d-474b-8fed-2c348d2ec8a5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:43.337892 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 839/1683] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:43 2022] PUT /identity/v3/projects/51f1a35e146e4e48a2c10bf15fd2070b/users/1aa5b84710304b9e8cdab672ccb5553b/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:43.339890 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3b5c9-641d-4099-bdc6-418a6d5a2eae None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:43.340186 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3b5c9-641d-4099-bdc6-418a6d5a2eae None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:43.341173 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3b5c9-641d-4099-bdc6-418a6d5a2eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:43.358385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:43.400598 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-eaf3b5c9-641d-4099-bdc6-418a6d5a2eae None None] MFA Rules not processed for user `1aa5b84710304b9e8cdab672ccb5553b`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:43.404275 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-eaf3b5c9-641d-4099-bdc6-418a6d5a2eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:43.422854 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 845/1684] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:43.427932 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db6878-63b8-4c74-a261-768dc85a900c None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:43.444120 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db6878-63b8-4c74-a261-768dc85a900c None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:43.545430 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe3057a-e287-4450-baaa-25b4b67c4a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:43.546999 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3057a-e287-4450-baaa-25b4b67c4a51 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:43.548241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3057a-e287-4450-baaa-25b4b67c4a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:43.548637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe3057a-e287-4450-baaa-25b4b67c4a51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:43.549622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe3057a-e287-4450-baaa-25b4b67c4a51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:43.554170 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe3057a-e287-4450-baaa-25b4b67c4a51 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:43.559061 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 840/1685] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:43.619311 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 846/1686] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:43 2022] POST /identity/v3/projects => generated 385 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:43.623461 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3b5c9-641d-4099-bdc6-418a6d5a2eae None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:43.640580 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3b5c9-641d-4099-bdc6-418a6d5a2eae None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:43.669057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:43.778016 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe3057a-e287-4450-baaa-25b4b67c4a51 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:43.789373 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc66e3d-9165-44ef-9f1e-299b9ff17652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:43.790351 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc66e3d-9165-44ef-9f1e-299b9ff17652 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:43.790517 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc66e3d-9165-44ef-9f1e-299b9ff17652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:43.790758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc66e3d-9165-44ef-9f1e-299b9ff17652 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:43.791370 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc66e3d-9165-44ef-9f1e-299b9ff17652 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:43.793908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc66e3d-9165-44ef-9f1e-299b9ff17652 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:43.829510 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 841/1687] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:43 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:43.832681 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc66e3d-9165-44ef-9f1e-299b9ff17652 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:43.852020 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc66e3d-9165-44ef-9f1e-299b9ff17652 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:43.858159 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fc95e-693b-47d6-8e9e-29bc8437455c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:43.858874 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fc95e-693b-47d6-8e9e-29bc8437455c admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:43.859212 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fc95e-693b-47d6-8e9e-29bc8437455c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:43.859968 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fc95e-693b-47d6-8e9e-29bc8437455c admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:43.860707 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fc95e-693b-47d6-8e9e-29bc8437455c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:43.863134 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fc95e-693b-47d6-8e9e-29bc8437455c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:43.883597 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 842/1688] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:43.888359 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fc95e-693b-47d6-8e9e-29bc8437455c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:43.914845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fc95e-693b-47d6-8e9e-29bc8437455c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:43.922049 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe3057a-e287-4450-baaa-25b4b67c4a51 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:43.929933 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f601c-8aec-49cf-81ac-3179f7fc56eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:43.931105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f601c-8aec-49cf-81ac-3179f7fc56eb admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:43.931902 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f601c-8aec-49cf-81ac-3179f7fc56eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:43.932853 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f601c-8aec-49cf-81ac-3179f7fc56eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:43.933105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e619c-b807-470e-aa0e-397b0a9e2d38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:43.934431 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e619c-b807-470e-aa0e-397b0a9e2d38 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:43.934885 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f601c-8aec-49cf-81ac-3179f7fc56eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:43.935343 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e619c-b807-470e-aa0e-397b0a9e2d38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:43.936983 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e619c-b807-470e-aa0e-397b0a9e2d38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:43.938519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f601c-8aec-49cf-81ac-3179f7fc56eb admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:43.964038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 843/1689] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:36:43.978745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f601c-8aec-49cf-81ac-3179f7fc56eb admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:44.007500 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f601c-8aec-49cf-81ac-3179f7fc56eb admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:44.021676 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106cffb-92f1-48ce-9fb1-933da85b642b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:44.022576 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106cffb-92f1-48ce-9fb1-933da85b642b admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:44.023181 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106cffb-92f1-48ce-9fb1-933da85b642b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:44.023506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106cffb-92f1-48ce-9fb1-933da85b642b admin admin] PATH_INFO: `/v3/projects/5b5bccc178424568a83b8eedc8728948/users/70f0fd9b81bf4bc5b52ee05a63ab4a1d/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:44.057247 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e106cffb-92f1-48ce-9fb1-933da85b642b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b5bccc178424568a83b8eedc8728948, user_id=70f0fd9b81bf4bc5b52ee05a63ab4a1d, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:44.063883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e106cffb-92f1-48ce-9fb1-933da85b642b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:44.091346 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e619c-b807-470e-aa0e-397b0a9e2d38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:44.091346 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e619c-b807-470e-aa0e-397b0a9e2d38 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:44.095595 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 844/1690] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:43 2022] PUT /identity/v3/projects/5b5bccc178424568a83b8eedc8728948/users/70f0fd9b81bf4bc5b52ee05a63ab4a1d/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:44.098524 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474aeb1-acfb-4723-8cb0-6e6a05d1275d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:44.099193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474aeb1-acfb-4723-8cb0-6e6a05d1275d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:44.099595 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474aeb1-acfb-4723-8cb0-6e6a05d1275d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:44.108967 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 847/1691] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:36:44.133652 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0474aeb1-acfb-4723-8cb0-6e6a05d1275d None None] MFA Rules not processed for user `70f0fd9b81bf4bc5b52ee05a63ab4a1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:44.134936 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0474aeb1-acfb-4723-8cb0-6e6a05d1275d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:44.204791 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e619c-b807-470e-aa0e-397b0a9e2d38 service cinder] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:44.216417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e619c-b807-470e-aa0e-397b0a9e2d38 service cinder] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:44.230740 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:44.232490 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 845/1692] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:44.235887 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474aeb1-acfb-4723-8cb0-6e6a05d1275d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:44.293777 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335a619-a670-4b6e-a51a-38f4451dc03a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:44.294376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335a619-a670-4b6e-a51a-38f4451dc03a service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:44.294683 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335a619-a670-4b6e-a51a-38f4451dc03a service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:44.295025 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335a619-a670-4b6e-a51a-38f4451dc03a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:44.301738 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:44.319838 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474aeb1-acfb-4723-8cb0-6e6a05d1275d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:44.326148 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6256f-cbb8-4f7e-a96a-9be80d91b29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:44.326812 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6256f-cbb8-4f7e-a96a-9be80d91b29e admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:44.327135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6256f-cbb8-4f7e-a96a-9be80d91b29e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:44.327462 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6256f-cbb8-4f7e-a96a-9be80d91b29e admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:44.328182 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6256f-cbb8-4f7e-a96a-9be80d91b29e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:44.330754 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6256f-cbb8-4f7e-a96a-9be80d91b29e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:44.368813 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 846/1693] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:44 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:36:44.372505 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6256f-cbb8-4f7e-a96a-9be80d91b29e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:44.374242 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9335a619-a670-4b6e-a51a-38f4451dc03a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:44.375806 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9335a619-a670-4b6e-a51a-38f4451dc03a service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:44.387394 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 848/1694] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:36:44.450966 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6256f-cbb8-4f7e-a96a-9be80d91b29e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:44.456298 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d30e3d-23f6-49af-8500-705154e207c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:44.457036 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d30e3d-23f6-49af-8500-705154e207c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:44.457241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d30e3d-23f6-49af-8500-705154e207c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:44.457445 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d30e3d-23f6-49af-8500-705154e207c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:44.458087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d30e3d-23f6-49af-8500-705154e207c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:44.460024 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d30e3d-23f6-49af-8500-705154e207c3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:44.485032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 847/1695] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:44 2022] POST /identity/v3/users => generated 298 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:36:44.487386 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335a619-a670-4b6e-a51a-38f4451dc03a service cinder] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:44.500383 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335a619-a670-4b6e-a51a-38f4451dc03a service cinder] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:44.521688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e6973-b991-43ad-94ab-4f8cd45fffd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:44.522410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e6973-b991-43ad-94ab-4f8cd45fffd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:44.522786 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e6973-b991-43ad-94ab-4f8cd45fffd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:44.523293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e6973-b991-43ad-94ab-4f8cd45fffd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:44.524153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470e6973-b991-43ad-94ab-4f8cd45fffd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:44.526089 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470e6973-b991-43ad-94ab-4f8cd45fffd1 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:44.542993 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d30e3d-23f6-49af-8500-705154e207c3 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:44.550785 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 849/1696] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:44.557592 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d30e3d-23f6-49af-8500-705154e207c3 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:44.565563 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c7659-ebf3-4495-9165-88809a8ee817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:44.566082 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c7659-ebf3-4495-9165-88809a8ee817 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:44.566453 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c7659-ebf3-4495-9165-88809a8ee817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:44.566821 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c7659-ebf3-4495-9165-88809a8ee817 admin admin] PATH_INFO: `/v3/projects/eda04a1c1fe54ca1a78cb5b27cef211e/users/3c35c7cbe30c4da0818ba2ea95bfcb2b/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:44.585796 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c7659-ebf3-4495-9165-88809a8ee817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eda04a1c1fe54ca1a78cb5b27cef211e, user_id=3c35c7cbe30c4da0818ba2ea95bfcb2b, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:44.588676 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c7659-ebf3-4495-9165-88809a8ee817 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:44.603834 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 848/1697] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:44 2022] PUT /identity/v3/projects/eda04a1c1fe54ca1a78cb5b27cef211e/users/3c35c7cbe30c4da0818ba2ea95bfcb2b/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:44.607866 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e6973-b991-43ad-94ab-4f8cd45fffd1 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:44.618160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e6973-b991-43ad-94ab-4f8cd45fffd1 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:44.681657 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91dc14c-1019-4a1f-9be5-91eb4fe7c89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:44.682268 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91dc14c-1019-4a1f-9be5-91eb4fe7c89e admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:44.682552 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91dc14c-1019-4a1f-9be5-91eb4fe7c89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:44.682826 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91dc14c-1019-4a1f-9be5-91eb4fe7c89e admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:44.683461 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91dc14c-1019-4a1f-9be5-91eb4fe7c89e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:44.685093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91dc14c-1019-4a1f-9be5-91eb4fe7c89e admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:44.695447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 850/1698] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:44.699382 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c7659-ebf3-4495-9165-88809a8ee817 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:44.710505 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c7659-ebf3-4495-9165-88809a8ee817 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:44.715077 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f492f12-32c3-4f12-aa68-b3557fa7d51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:44.715718 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f492f12-32c3-4f12-aa68-b3557fa7d51f admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:44.715996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f492f12-32c3-4f12-aa68-b3557fa7d51f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:44.716266 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f492f12-32c3-4f12-aa68-b3557fa7d51f admin admin] PATH_INFO: `/v3/projects/eda04a1c1fe54ca1a78cb5b27cef211e/users/3c35c7cbe30c4da0818ba2ea95bfcb2b/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:44.718155 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f492f12-32c3-4f12-aa68-b3557fa7d51f admin admin] RBAC: Authorizing `identity:create_grant(project_id=eda04a1c1fe54ca1a78cb5b27cef211e, user_id=3c35c7cbe30c4da0818ba2ea95bfcb2b, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:44.720872 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f492f12-32c3-4f12-aa68-b3557fa7d51f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:44.730720 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 849/1699] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:44 2022] PUT /identity/v3/projects/eda04a1c1fe54ca1a78cb5b27cef211e/users/3c35c7cbe30c4da0818ba2ea95bfcb2b/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:36:44.741640 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03ae0f-949f-43ea-aa50-dd55693f31de None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:44.741928 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03ae0f-949f-43ea-aa50-dd55693f31de None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:44.742231 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03ae0f-949f-43ea-aa50-dd55693f31de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:44.770868 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-cc03ae0f-949f-43ea-aa50-dd55693f31de None None] MFA Rules not processed for user `3c35c7cbe30c4da0818ba2ea95bfcb2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:44.771982 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-cc03ae0f-949f-43ea-aa50-dd55693f31de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:44.811736 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:44.839391 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 851/1700] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:44 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:44.850617 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f492f12-32c3-4f12-aa68-b3557fa7d51f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:44.906777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f492f12-32c3-4f12-aa68-b3557fa7d51f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:44.910938 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ad1b4-f6cf-45e7-bf27-f1732fe70014 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:44.911634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ad1b4-f6cf-45e7-bf27-f1732fe70014 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:44.911809 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ad1b4-f6cf-45e7-bf27-f1732fe70014 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:44.912082 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ad1b4-f6cf-45e7-bf27-f1732fe70014 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:44.952839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:44.970927 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ad1b4-f6cf-45e7-bf27-f1732fe70014 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:44.972527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ad1b4-f6cf-45e7-bf27-f1732fe70014 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:44.982415 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 850/1701] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:46.566343 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03ae0f-949f-43ea-aa50-dd55693f31de None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:46.568039 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-cc03ae0f-949f-43ea-aa50-dd55693f31de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:46.579690 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03ae0f-949f-43ea-aa50-dd55693f31de None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:46.584164 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf161d7-5db5-4e7b-a1bd-47402847e9cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:46.584771 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf161d7-5db5-4e7b-a1bd-47402847e9cc service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:46.585059 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf161d7-5db5-4e7b-a1bd-47402847e9cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:46.585333 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf161d7-5db5-4e7b-a1bd-47402847e9cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:46.668679 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf161d7-5db5-4e7b-a1bd-47402847e9cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:46.670584 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf161d7-5db5-4e7b-a1bd-47402847e9cc service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:46.681697 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 852/1702] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:47.165614 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ad1b4-f6cf-45e7-bf27-f1732fe70014 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:47.231704 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ad1b4-f6cf-45e7-bf27-f1732fe70014 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:47.236432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0649cf-1d14-4fcc-9815-c106b22d5b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:47.237662 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0649cf-1d14-4fcc-9815-c106b22d5b5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:47.237962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0649cf-1d14-4fcc-9815-c106b22d5b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:47.238230 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0649cf-1d14-4fcc-9815-c106b22d5b5e admin admin] PATH_INFO: `/v3/users/ad55bfa477224aa38bd067ab12ec2c81` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:47.239352 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0649cf-1d14-4fcc-9815-c106b22d5b5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad55bfa477224aa38bd067ab12ec2c81)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:47.241324 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0649cf-1d14-4fcc-9815-c106b22d5b5e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:47.300517 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6e0649cf-1d14-4fcc-9815-c106b22d5b5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad55bfa477224aa38bd067ab12ec2c81'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:47.300517 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6e0649cf-1d14-4fcc-9815-c106b22d5b5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad55bfa477224aa38bd067ab12ec2c81'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:47.307499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6e0649cf-1d14-4fcc-9815-c106b22d5b5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad55bfa477224aa38bd067ab12ec2c81'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:47.308439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf161d7-5db5-4e7b-a1bd-47402847e9cc service cinder] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:47.320277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6e0649cf-1d14-4fcc-9815-c106b22d5b5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad55bfa477224aa38bd067ab12ec2c81'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:47.320277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6e0649cf-1d14-4fcc-9815-c106b22d5b5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad55bfa477224aa38bd067ab12ec2c81'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:47.320532 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf161d7-5db5-4e7b-a1bd-47402847e9cc service cinder] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:47.328012 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904f768-e490-4b9a-801a-1a3036199a92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:47.328012 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904f768-e490-4b9a-801a-1a3036199a92 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:47.328012 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904f768-e490-4b9a-801a-1a3036199a92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:47.328012 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904f768-e490-4b9a-801a-1a3036199a92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:47.328371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-3904f768-e490-4b9a-801a-1a3036199a92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:47.338381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3904f768-e490-4b9a-801a-1a3036199a92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:47.340083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3904f768-e490-4b9a-801a-1a3036199a92 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:47.348797 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 851/1703] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:47 2022] DELETE /identity/v3/users/ad55bfa477224aa38bd067ab12ec2c81 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:47.352177 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 853/1704] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:36:47.353300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46872260-31c1-4f98-bd6c-032b14435a5c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:47.353639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46872260-31c1-4f98-bd6c-032b14435a5c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:47.353916 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46872260-31c1-4f98-bd6c-032b14435a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:47.372938 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-46872260-31c1-4f98-bd6c-032b14435a5c None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:47.374139 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-46872260-31c1-4f98-bd6c-032b14435a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:47.425383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:47.435150 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 852/1705] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:36:47.445351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904f768-e490-4b9a-801a-1a3036199a92 service cinder] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:47.446785 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-3904f768-e490-4b9a-801a-1a3036199a92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:47.515872 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904f768-e490-4b9a-801a-1a3036199a92 service cinder] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:47.564663 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d8d9c-abf3-4b80-bd2d-d2adb287207b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:47.565424 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d8d9c-abf3-4b80-bd2d-d2adb287207b service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:47.565706 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d8d9c-abf3-4b80-bd2d-d2adb287207b service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:47.566004 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d8d9c-abf3-4b80-bd2d-d2adb287207b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:47.574911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d8d9c-abf3-4b80-bd2d-d2adb287207b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:47.576349 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d8d9c-abf3-4b80-bd2d-d2adb287207b service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:47.586396 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 854/1706] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:47.857511 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46872260-31c1-4f98-bd6c-032b14435a5c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:47.859006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-46872260-31c1-4f98-bd6c-032b14435a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:47.872844 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46872260-31c1-4f98-bd6c-032b14435a5c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:47.878525 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b898e0-7039-48a4-a0e2-b4f30c446ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:47.879166 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b898e0-7039-48a4-a0e2-b4f30c446ffb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:47.879441 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b898e0-7039-48a4-a0e2-b4f30c446ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:47.879711 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b898e0-7039-48a4-a0e2-b4f30c446ffb admin admin] PATH_INFO: `/v3/projects/190e4a2e98cd41f89df5e698bd45bbe4` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:47.880820 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b898e0-7039-48a4-a0e2-b4f30c446ffb admin admin] RBAC: Authorizing `identity:delete_project(project_id=190e4a2e98cd41f89df5e698bd45bbe4)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:47.883515 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b898e0-7039-48a4-a0e2-b4f30c446ffb admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:47.934390 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-d4b898e0-7039-48a4-a0e2-b4f30c446ffb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '190e4a2e98cd41f89df5e698bd45bbe4', 'request_id': 'req-d4b898e0-7039-48a4-a0e2-b4f30c446ffb'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:47.941112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-d4b898e0-7039-48a4-a0e2-b4f30c446ffb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '190e4a2e98cd41f89df5e698bd45bbe4', 'request_id': 'req-d4b898e0-7039-48a4-a0e2-b4f30c446ffb'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:47.948722 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-d4b898e0-7039-48a4-a0e2-b4f30c446ffb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '190e4a2e98cd41f89df5e698bd45bbe4', 'request_id': 'req-d4b898e0-7039-48a4-a0e2-b4f30c446ffb'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:47.960281 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-d4b898e0-7039-48a4-a0e2-b4f30c446ffb admin admin] The token cache is being invalidate because project 190e4a2e98cd41f89df5e698bd45bbe4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:47.960571 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-d4b898e0-7039-48a4-a0e2-b4f30c446ffb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:47.961221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-d4b898e0-7039-48a4-a0e2-b4f30c446ffb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:47.962687 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 853/1707] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:47 2022] DELETE /identity/v3/projects/190e4a2e98cd41f89df5e698bd45bbe4 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:47.966588 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d8d9c-abf3-4b80-bd2d-d2adb287207b service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:47.978150 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d8d9c-abf3-4b80-bd2d-d2adb287207b service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:47.999450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:48.049820 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676d645-8f2e-4917-a0f5-2bf3fb4c9b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:48.050469 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676d645-8f2e-4917-a0f5-2bf3fb4c9b6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:48.050747 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676d645-8f2e-4917-a0f5-2bf3fb4c9b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:48.051216 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676d645-8f2e-4917-a0f5-2bf3fb4c9b6d admin admin] PATH_INFO: `/v3/users/1aa5b84710304b9e8cdab672ccb5553b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:48.052350 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d676d645-8f2e-4917-a0f5-2bf3fb4c9b6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aa5b84710304b9e8cdab672ccb5553b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:48.054328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d676d645-8f2e-4917-a0f5-2bf3fb4c9b6d admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:48.104315 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b898e0-7039-48a4-a0e2-b4f30c446ffb admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:48.105782 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d4b898e0-7039-48a4-a0e2-b4f30c446ffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:48.113891 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-d676d645-8f2e-4917-a0f5-2bf3fb4c9b6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aa5b84710304b9e8cdab672ccb5553b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:48.115206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-d676d645-8f2e-4917-a0f5-2bf3fb4c9b6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aa5b84710304b9e8cdab672ccb5553b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:48.123498 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-d676d645-8f2e-4917-a0f5-2bf3fb4c9b6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aa5b84710304b9e8cdab672ccb5553b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:48.124441 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-d676d645-8f2e-4917-a0f5-2bf3fb4c9b6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aa5b84710304b9e8cdab672ccb5553b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:48.132983 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-d676d645-8f2e-4917-a0f5-2bf3fb4c9b6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aa5b84710304b9e8cdab672ccb5553b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:48.158430 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 855/1708] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:47 2022] DELETE /identity/v3/users/1aa5b84710304b9e8cdab672ccb5553b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:48.173662 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b898e0-7039-48a4-a0e2-b4f30c446ffb admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:48.230339 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af963b-ff15-4409-803a-ed218590b77c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:48.230959 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af963b-ff15-4409-803a-ed218590b77c service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:48.231239 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af963b-ff15-4409-803a-ed218590b77c service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:48.231643 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af963b-ff15-4409-803a-ed218590b77c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:48.233744 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-04af963b-ff15-4409-803a-ed218590b77c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:48.298208 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:48.319993 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af963b-ff15-4409-803a-ed218590b77c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:48.321678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af963b-ff15-4409-803a-ed218590b77c service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:48.333588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 854/1709] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:48 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:36:48.517597 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676d645-8f2e-4917-a0f5-2bf3fb4c9b6d admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:48.520566 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-d676d645-8f2e-4917-a0f5-2bf3fb4c9b6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:48.637163 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676d645-8f2e-4917-a0f5-2bf3fb4c9b6d admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:48.637163 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e5f21-e91e-4a49-9634-44344ba5e63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:48.637163 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e5f21-e91e-4a49-9634-44344ba5e63c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:48.637163 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e5f21-e91e-4a49-9634-44344ba5e63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:48.638030 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e5f21-e91e-4a49-9634-44344ba5e63c admin admin] PATH_INFO: `/v3/projects/51f1a35e146e4e48a2c10bf15fd2070b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:48.638030 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e5f21-e91e-4a49-9634-44344ba5e63c admin admin] RBAC: Authorizing `identity:delete_project(project_id=51f1a35e146e4e48a2c10bf15fd2070b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:48.638030 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e5f21-e91e-4a49-9634-44344ba5e63c admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:48.709079 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:48.713800 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-774e5f21-e91e-4a49-9634-44344ba5e63c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51f1a35e146e4e48a2c10bf15fd2070b', 'request_id': 'req-774e5f21-e91e-4a49-9634-44344ba5e63c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:48.723725 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-774e5f21-e91e-4a49-9634-44344ba5e63c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51f1a35e146e4e48a2c10bf15fd2070b', 'request_id': 'req-774e5f21-e91e-4a49-9634-44344ba5e63c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:48.733300 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-774e5f21-e91e-4a49-9634-44344ba5e63c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51f1a35e146e4e48a2c10bf15fd2070b', 'request_id': 'req-774e5f21-e91e-4a49-9634-44344ba5e63c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:48.745867 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-774e5f21-e91e-4a49-9634-44344ba5e63c admin admin] The token cache is being invalidate because project 51f1a35e146e4e48a2c10bf15fd2070b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:48.746233 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-774e5f21-e91e-4a49-9634-44344ba5e63c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:48.747063 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-774e5f21-e91e-4a49-9634-44344ba5e63c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:48.748798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 856/1710] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:48 2022] DELETE /identity/v3/projects/51f1a35e146e4e48a2c10bf15fd2070b => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:48.754647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324da767-c370-41f2-972b-afb45d658113 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:48.755147 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324da767-c370-41f2-972b-afb45d658113 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:48.755435 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324da767-c370-41f2-972b-afb45d658113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:48.779520 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-324da767-c370-41f2-972b-afb45d658113 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:48.780732 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-324da767-c370-41f2-972b-afb45d658113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:48.862603 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 855/1711] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:48.870410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9064c-8b2e-4a6a-8d7c-4d7abd3d1422 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:48.870717 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9064c-8b2e-4a6a-8d7c-4d7abd3d1422 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:48.870994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9064c-8b2e-4a6a-8d7c-4d7abd3d1422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:48.892034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-9ea9064c-8b2e-4a6a-8d7c-4d7abd3d1422 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:48.893532 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-9ea9064c-8b2e-4a6a-8d7c-4d7abd3d1422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:48.901492 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 857/1712] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:48.905663 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324da767-c370-41f2-972b-afb45d658113 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:48.915504 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324da767-c370-41f2-972b-afb45d658113 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:48.920134 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84a944-c9bf-4719-84f3-597bd66d1148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:48.920731 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84a944-c9bf-4719-84f3-597bd66d1148 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:48.921053 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84a944-c9bf-4719-84f3-597bd66d1148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:48.921378 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84a944-c9bf-4719-84f3-597bd66d1148 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:48.922018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84a944-c9bf-4719-84f3-597bd66d1148 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:36:48.922328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84a944-c9bf-4719-84f3-597bd66d1148 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:48.923887 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84a944-c9bf-4719-84f3-597bd66d1148 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:48.936256 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 856/1713] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:36:48.941282 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea9064c-8b2e-4a6a-8d7c-4d7abd3d1422 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:48.950844 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea9064c-8b2e-4a6a-8d7c-4d7abd3d1422 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:48.956964 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ca00d-26fe-4c79-b523-6d80b3ad2c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:48.957569 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ca00d-26fe-4c79-b523-6d80b3ad2c2c admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:48.957851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ca00d-26fe-4c79-b523-6d80b3ad2c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:48.958125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ca00d-26fe-4c79-b523-6d80b3ad2c2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:48.958760 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ca00d-26fe-4c79-b523-6d80b3ad2c2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:48.960669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ca00d-26fe-4c79-b523-6d80b3ad2c2c admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:48.987904 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 858/1714] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:48 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:36:48.992132 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84a944-c9bf-4719-84f3-597bd66d1148 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:49.002794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84a944-c9bf-4719-84f3-597bd66d1148 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:49.059379 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6553f5f-5946-4e2d-ba69-0532d360578a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:49.059994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6553f5f-5946-4e2d-ba69-0532d360578a admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:49.060276 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6553f5f-5946-4e2d-ba69-0532d360578a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:49.060602 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6553f5f-5946-4e2d-ba69-0532d360578a admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:49.061300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6553f5f-5946-4e2d-ba69-0532d360578a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:49.063221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6553f5f-5946-4e2d-ba69-0532d360578a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:49.085861 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 857/1715] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:48 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:36:49.089987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ca00d-26fe-4c79-b523-6d80b3ad2c2c admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:49.101491 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ca00d-26fe-4c79-b523-6d80b3ad2c2c admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:49.107509 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1ee64-312f-41f1-8aa1-2f1fcb8d74aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:49.108282 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1ee64-312f-41f1-8aa1-2f1fcb8d74aa admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:49.108565 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1ee64-312f-41f1-8aa1-2f1fcb8d74aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:49.108849 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1ee64-312f-41f1-8aa1-2f1fcb8d74aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:49.109619 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1ee64-312f-41f1-8aa1-2f1fcb8d74aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:49.111127 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1ee64-312f-41f1-8aa1-2f1fcb8d74aa admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:49.123568 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 859/1716] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:49.127118 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6553f5f-5946-4e2d-ba69-0532d360578a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:49.137751 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6553f5f-5946-4e2d-ba69-0532d360578a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:49.142632 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3656c-cdbd-4d73-a11a-e7639beedf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:49.143853 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3656c-cdbd-4d73-a11a-e7639beedf66 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:49.144253 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3656c-cdbd-4d73-a11a-e7639beedf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:49.144532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3656c-cdbd-4d73-a11a-e7639beedf66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:49.145209 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3656c-cdbd-4d73-a11a-e7639beedf66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:49.146804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3656c-cdbd-4d73-a11a-e7639beedf66 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:49.163161 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 858/1717] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:49.164565 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1ee64-312f-41f1-8aa1-2f1fcb8d74aa admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:49.175338 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1ee64-312f-41f1-8aa1-2f1fcb8d74aa admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:49.179985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6a7d2-e6d9-45a4-b498-c1820ec7797d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:49.180632 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6a7d2-e6d9-45a4-b498-c1820ec7797d admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:49.180946 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6a7d2-e6d9-45a4-b498-c1820ec7797d admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:49.181222 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6a7d2-e6d9-45a4-b498-c1820ec7797d admin admin] PATH_INFO: `/v3/projects/b0f76b7251e54367a19e8a0a04699181/users/fdb2865b94774eebb327d43dbae386c2/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:49.197737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6a7d2-e6d9-45a4-b498-c1820ec7797d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f76b7251e54367a19e8a0a04699181, user_id=fdb2865b94774eebb327d43dbae386c2, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:49.200375 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6a7d2-e6d9-45a4-b498-c1820ec7797d admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:49.214766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 860/1718] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:49 2022] PUT /identity/v3/projects/b0f76b7251e54367a19e8a0a04699181/users/fdb2865b94774eebb327d43dbae386c2/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:49.226601 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7cb56-f916-4651-9fde-d88dbf7510aa None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:49.227082 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7cb56-f916-4651-9fde-d88dbf7510aa None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:49.227169 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7cb56-f916-4651-9fde-d88dbf7510aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:49.255355 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-deb7cb56-f916-4651-9fde-d88dbf7510aa None None] MFA Rules not processed for user `fdb2865b94774eebb327d43dbae386c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:49.256544 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-deb7cb56-f916-4651-9fde-d88dbf7510aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:49.300209 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:49.321361 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 859/1719] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:49.326066 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6a7d2-e6d9-45a4-b498-c1820ec7797d admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:49.390015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6a7d2-e6d9-45a4-b498-c1820ec7797d admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:49.394795 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436a5b8-a312-4d54-bf37-b73d00ef7e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:49.395395 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436a5b8-a312-4d54-bf37-b73d00ef7e33 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:49.395671 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436a5b8-a312-4d54-bf37-b73d00ef7e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:49.395942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436a5b8-a312-4d54-bf37-b73d00ef7e33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:49.396574 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436a5b8-a312-4d54-bf37-b73d00ef7e33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:49.398492 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436a5b8-a312-4d54-bf37-b73d00ef7e33 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:49.426259 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 861/1720] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:49 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:49.430074 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7cb56-f916-4651-9fde-d88dbf7510aa None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:49.440941 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7cb56-f916-4651-9fde-d88dbf7510aa None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:49.501063 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d7994-ce76-4459-9a18-e6872fe4a960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:49.501758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d7994-ce76-4459-9a18-e6872fe4a960 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:49.502278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d7994-ce76-4459-9a18-e6872fe4a960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:49.502880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d7994-ce76-4459-9a18-e6872fe4a960 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:49.503604 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d7994-ce76-4459-9a18-e6872fe4a960 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:49.505775 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d7994-ce76-4459-9a18-e6872fe4a960 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:49.536917 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 860/1721] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:49 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:36:49.538426 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436a5b8-a312-4d54-bf37-b73d00ef7e33 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:49.549704 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436a5b8-a312-4d54-bf37-b73d00ef7e33 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:49.555222 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b53302-ff9f-4e55-989e-6ae2cf4da9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:49.555841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b53302-ff9f-4e55-989e-6ae2cf4da9b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:49.556389 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b53302-ff9f-4e55-989e-6ae2cf4da9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:49.556618 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b53302-ff9f-4e55-989e-6ae2cf4da9b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:49.557198 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b53302-ff9f-4e55-989e-6ae2cf4da9b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:49.558806 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b53302-ff9f-4e55-989e-6ae2cf4da9b2 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:49.571169 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 862/1722] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:49.575418 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d7994-ce76-4459-9a18-e6872fe4a960 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:49.586886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d7994-ce76-4459-9a18-e6872fe4a960 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:49.594874 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661ed6e-27ed-4a94-b68a-0da01fad1b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:49.596546 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661ed6e-27ed-4a94-b68a-0da01fad1b6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:49.596546 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661ed6e-27ed-4a94-b68a-0da01fad1b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:49.596546 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661ed6e-27ed-4a94-b68a-0da01fad1b6a admin admin] PATH_INFO: `/v3/projects/72f8dda61d3c4561ac5af3f496a39ad2/users/ba91c52ff3e94bc0b98b19c8fe18f349/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:49.612954 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2661ed6e-27ed-4a94-b68a-0da01fad1b6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=72f8dda61d3c4561ac5af3f496a39ad2, user_id=ba91c52ff3e94bc0b98b19c8fe18f349, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:49.616174 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2661ed6e-27ed-4a94-b68a-0da01fad1b6a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:49.633856 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 861/1723] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:49 2022] PUT /identity/v3/projects/72f8dda61d3c4561ac5af3f496a39ad2/users/ba91c52ff3e94bc0b98b19c8fe18f349/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:49.637914 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b53302-ff9f-4e55-989e-6ae2cf4da9b2 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:49.649219 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b53302-ff9f-4e55-989e-6ae2cf4da9b2 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:49.691581 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:49.715200 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb518a-6ff7-4b33-99b9-cef249211f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:49.716153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb518a-6ff7-4b33-99b9-cef249211f22 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:49.716506 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb518a-6ff7-4b33-99b9-cef249211f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:49.716853 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb518a-6ff7-4b33-99b9-cef249211f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:49.717642 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb518a-6ff7-4b33-99b9-cef249211f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:49.719318 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb518a-6ff7-4b33-99b9-cef249211f22 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:49.738233 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 863/1724] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:36:49.742374 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661ed6e-27ed-4a94-b68a-0da01fad1b6a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:49.755171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661ed6e-27ed-4a94-b68a-0da01fad1b6a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:49.759923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb8795-9dd6-4cb4-b78d-2f50c9c259c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:49.760819 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb8795-9dd6-4cb4-b78d-2f50c9c259c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:49.761487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb8795-9dd6-4cb4-b78d-2f50c9c259c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:49.761983 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb8795-9dd6-4cb4-b78d-2f50c9c259c4 admin admin] PATH_INFO: `/v3/projects/72f8dda61d3c4561ac5af3f496a39ad2/users/ba91c52ff3e94bc0b98b19c8fe18f349/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:49.763919 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb8795-9dd6-4cb4-b78d-2f50c9c259c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72f8dda61d3c4561ac5af3f496a39ad2, user_id=ba91c52ff3e94bc0b98b19c8fe18f349, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:49.766724 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb8795-9dd6-4cb4-b78d-2f50c9c259c4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:49.784903 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 862/1725] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:49 2022] PUT /identity/v3/projects/72f8dda61d3c4561ac5af3f496a39ad2/users/ba91c52ff3e94bc0b98b19c8fe18f349/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:49.790079 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83050978-3a23-4c8d-9e6b-6cefc61650ea None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:49.790368 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83050978-3a23-4c8d-9e6b-6cefc61650ea None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:49.790644 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83050978-3a23-4c8d-9e6b-6cefc61650ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:49.832943 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-83050978-3a23-4c8d-9e6b-6cefc61650ea None None] MFA Rules not processed for user `ba91c52ff3e94bc0b98b19c8fe18f349`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:49.834256 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-83050978-3a23-4c8d-9e6b-6cefc61650ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:49.924946 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 864/1726] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:49 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:49.967597 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb8795-9dd6-4cb4-b78d-2f50c9c259c4 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:49.969409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2bfb8795-9dd6-4cb4-b78d-2f50c9c259c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:50.045181 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb8795-9dd6-4cb4-b78d-2f50c9c259c4 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:50.048903 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24b4e2-392a-42f0-9a0f-fb4c71a8dba0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:50.050520 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24b4e2-392a-42f0-9a0f-fb4c71a8dba0 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:50.050883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24b4e2-392a-42f0-9a0f-fb4c71a8dba0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:50.051163 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24b4e2-392a-42f0-9a0f-fb4c71a8dba0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:50.060911 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24b4e2-392a-42f0-9a0f-fb4c71a8dba0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:50.062511 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24b4e2-392a-42f0-9a0f-fb4c71a8dba0 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:50.072671 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 863/1727] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:49 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:50.286945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e6195-6144-4c79-8ba0-a1b30f35b981 None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:50.287306 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e6195-6144-4c79-8ba0-a1b30f35b981 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:50.287669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e6195-6144-4c79-8ba0-a1b30f35b981 None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:50.290023 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 865/1728] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 20 13:36:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:36:50.323216 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86342e9-a5e0-46d6-bed9-717d182d9b0f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:50.323784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86342e9-a5e0-46d6-bed9-717d182d9b0f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:50.324048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86342e9-a5e0-46d6-bed9-717d182d9b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:50.356753 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a86342e9-a5e0-46d6-bed9-717d182d9b0f None None] MFA Rules not processed for user `2440594934b64621a01c441133778155`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:50.358490 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a86342e9-a5e0-46d6-bed9-717d182d9b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:50.425862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:50.468556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 864/1729] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 20 13:36:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:50.480505 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e6195-6144-4c79-8ba0-a1b30f35b981 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:50.482690 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-5a1e6195-6144-4c79-8ba0-a1b30f35b981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:50.525436 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e6195-6144-4c79-8ba0-a1b30f35b981 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:50.533718 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8c0a8e-563d-49c3-b456-78abf83e6083 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2440594934b64621a01c441133778155', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:50.534697 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c0a8e-563d-49c3-b456-78abf83e6083 service placement] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:50.535192 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c0a8e-563d-49c3-b456-78abf83e6083 service placement] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:50.535617 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c0a8e-563d-49c3-b456-78abf83e6083 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:50.547876 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8c0a8e-563d-49c3-b456-78abf83e6083 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:50.550496 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8c0a8e-563d-49c3-b456-78abf83e6083 service placement] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:50.562790 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 866/1730] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:50.871589 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86342e9-a5e0-46d6-bed9-717d182d9b0f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:50.939988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86342e9-a5e0-46d6-bed9-717d182d9b0f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:50.945310 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ac380-ef8e-4a36-ae37-29f5fde2c0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:50.946618 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ac380-ef8e-4a36-ae37-29f5fde2c0a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:50.946618 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ac380-ef8e-4a36-ae37-29f5fde2c0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:50.946618 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ac380-ef8e-4a36-ae37-29f5fde2c0a5 admin admin] PATH_INFO: `/v3/users/fdb2865b94774eebb327d43dbae386c2` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:50.947681 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ac380-ef8e-4a36-ae37-29f5fde2c0a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdb2865b94774eebb327d43dbae386c2)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:50.949644 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ac380-ef8e-4a36-ae37-29f5fde2c0a5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:51.005364 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a87ac380-ef8e-4a36-ae37-29f5fde2c0a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdb2865b94774eebb327d43dbae386c2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.006499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a87ac380-ef8e-4a36-ae37-29f5fde2c0a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdb2865b94774eebb327d43dbae386c2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.013922 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8c0a8e-563d-49c3-b456-78abf83e6083 service placement] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:51.026038 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-7b8c0a8e-563d-49c3-b456-78abf83e6083 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:51.030751 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a87ac380-ef8e-4a36-ae37-29f5fde2c0a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdb2865b94774eebb327d43dbae386c2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.038403 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a87ac380-ef8e-4a36-ae37-29f5fde2c0a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdb2865b94774eebb327d43dbae386c2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.039059 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a87ac380-ef8e-4a36-ae37-29f5fde2c0a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdb2865b94774eebb327d43dbae386c2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.041557 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8c0a8e-563d-49c3-b456-78abf83e6083 service placement] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:51.046168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a0aa1-3055-4699-a707-c84c00f74a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:51.046996 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a0aa1-3055-4699-a707-c84c00f74a65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:51.047280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a0aa1-3055-4699-a707-c84c00f74a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:51.047555 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a0aa1-3055-4699-a707-c84c00f74a65 admin admin] PATH_INFO: `/v3/users/a9e063603e0544468f8f610c8b3a52ee` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:51.048680 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a0aa1-3055-4699-a707-c84c00f74a65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9e063603e0544468f8f610c8b3a52ee)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:51.051273 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a0aa1-3055-4699-a707-c84c00f74a65 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:51.057739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 865/1731] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:50 2022] DELETE /identity/v3/users/fdb2865b94774eebb327d43dbae386c2 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:51.062141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719e93b-3526-429a-a29c-5ed4dba078fd None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:51.062427 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719e93b-3526-429a-a29c-5ed4dba078fd None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:51.062706 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719e93b-3526-429a-a29c-5ed4dba078fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:51.082374 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a719e93b-3526-429a-a29c-5ed4dba078fd None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:51.083576 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a719e93b-3526-429a-a29c-5ed4dba078fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:51.098616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-b86a0aa1-3055-4699-a707-c84c00f74a65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9e063603e0544468f8f610c8b3a52ee'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.099341 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-b86a0aa1-3055-4699-a707-c84c00f74a65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9e063603e0544468f8f610c8b3a52ee'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.107671 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-b86a0aa1-3055-4699-a707-c84c00f74a65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9e063603e0544468f8f610c8b3a52ee'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.108416 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-b86a0aa1-3055-4699-a707-c84c00f74a65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9e063603e0544468f8f610c8b3a52ee'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.117271 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-b86a0aa1-3055-4699-a707-c84c00f74a65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9e063603e0544468f8f610c8b3a52ee'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.135969 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:51.136391 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 867/1732] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:51 2022] DELETE /identity/v3/users/a9e063603e0544468f8f610c8b3a52ee => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:51.153510 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 866/1733] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:36:51.164152 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a0aa1-3055-4699-a707-c84c00f74a65 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:51.165914 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b86a0aa1-3055-4699-a707-c84c00f74a65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:51.204644 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:51.245324 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a0aa1-3055-4699-a707-c84c00f74a65 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:51.249441 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36683f6-d4f7-4f9e-a0a5-3d4ed9bfff3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:51.250069 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36683f6-d4f7-4f9e-a0a5-3d4ed9bfff3e service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:51.250348 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36683f6-d4f7-4f9e-a0a5-3d4ed9bfff3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:51.250621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36683f6-d4f7-4f9e-a0a5-3d4ed9bfff3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:51.324856 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36683f6-d4f7-4f9e-a0a5-3d4ed9bfff3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:51.326599 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36683f6-d4f7-4f9e-a0a5-3d4ed9bfff3e service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:51.333269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719e93b-3526-429a-a29c-5ed4dba078fd None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:51.334807 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a719e93b-3526-429a-a29c-5ed4dba078fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:51.337892 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 868/1734] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:51.347242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719e93b-3526-429a-a29c-5ed4dba078fd None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:51.351968 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c499e1-65a7-4b1a-b13b-515786f601c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:51.352692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c499e1-65a7-4b1a-b13b-515786f601c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:51.353002 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c499e1-65a7-4b1a-b13b-515786f601c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:51.353280 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c499e1-65a7-4b1a-b13b-515786f601c8 admin admin] PATH_INFO: `/v3/projects/22424a637585415d86db35e23e1e06fe` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:51.354408 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c499e1-65a7-4b1a-b13b-515786f601c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22424a637585415d86db35e23e1e06fe)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:51.356698 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c499e1-65a7-4b1a-b13b-515786f601c8 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:51.409527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-81c499e1-65a7-4b1a-b13b-515786f601c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22424a637585415d86db35e23e1e06fe', 'request_id': 'req-81c499e1-65a7-4b1a-b13b-515786f601c8'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.416595 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-81c499e1-65a7-4b1a-b13b-515786f601c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22424a637585415d86db35e23e1e06fe', 'request_id': 'req-81c499e1-65a7-4b1a-b13b-515786f601c8'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.426708 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-81c499e1-65a7-4b1a-b13b-515786f601c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22424a637585415d86db35e23e1e06fe', 'request_id': 'req-81c499e1-65a7-4b1a-b13b-515786f601c8'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.439347 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-81c499e1-65a7-4b1a-b13b-515786f601c8 admin admin] The token cache is being invalidate because project 22424a637585415d86db35e23e1e06fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:51.440492 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-81c499e1-65a7-4b1a-b13b-515786f601c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.441177 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-81c499e1-65a7-4b1a-b13b-515786f601c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.443439 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 867/1735] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:51 2022] DELETE /identity/v3/projects/22424a637585415d86db35e23e1e06fe => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:36:51.449420 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36683f6-d4f7-4f9e-a0a5-3d4ed9bfff3e service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:51.459753 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36683f6-d4f7-4f9e-a0a5-3d4ed9bfff3e service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:51.526977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c499e1-65a7-4b1a-b13b-515786f601c8 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:51.528658 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-81c499e1-65a7-4b1a-b13b-515786f601c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:51.554866 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995d52f-e183-4640-80f4-7f83a1eb9cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:51.555942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995d52f-e183-4640-80f4-7f83a1eb9cb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:51.555942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995d52f-e183-4640-80f4-7f83a1eb9cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:51.556279 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995d52f-e183-4640-80f4-7f83a1eb9cb8 admin admin] PATH_INFO: `/v3/users/4e6fe9efa7f8487b96370ab4f99dc21c` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:51.557925 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995d52f-e183-4640-80f4-7f83a1eb9cb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e6fe9efa7f8487b96370ab4f99dc21c)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:51.560522 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995d52f-e183-4640-80f4-7f83a1eb9cb8 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:51.603312 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c499e1-65a7-4b1a-b13b-515786f601c8 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:51.608433 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc9ce3-83f4-4d09-8663-cfd9b8294744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:51.609175 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc9ce3-83f4-4d09-8663-cfd9b8294744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:51.609523 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc9ce3-83f4-4d09-8663-cfd9b8294744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:51.609865 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc9ce3-83f4-4d09-8663-cfd9b8294744 admin admin] PATH_INFO: `/v3/users/7d552a2f6d5649afb1a363a8b6fd611d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:51.611328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfc9ce3-83f4-4d09-8663-cfd9b8294744 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d552a2f6d5649afb1a363a8b6fd611d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:51.612902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-e995d52f-e183-4640-80f4-7f83a1eb9cb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e6fe9efa7f8487b96370ab4f99dc21c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.613451 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfc9ce3-83f4-4d09-8663-cfd9b8294744 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:51.613911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-e995d52f-e183-4640-80f4-7f83a1eb9cb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e6fe9efa7f8487b96370ab4f99dc21c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.622522 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-e995d52f-e183-4640-80f4-7f83a1eb9cb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e6fe9efa7f8487b96370ab4f99dc21c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.623992 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-e995d52f-e183-4640-80f4-7f83a1eb9cb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e6fe9efa7f8487b96370ab4f99dc21c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.635682 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-e995d52f-e183-4640-80f4-7f83a1eb9cb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e6fe9efa7f8487b96370ab4f99dc21c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.646273 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:51.666764 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 869/1736] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:51 2022] DELETE /identity/v3/users/4e6fe9efa7f8487b96370ab4f99dc21c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:51.669673 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995d52f-e183-4640-80f4-7f83a1eb9cb8 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:51.669673 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-e995d52f-e183-4640-80f4-7f83a1eb9cb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:51.692283 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-bcfc9ce3-83f4-4d09-8663-cfd9b8294744 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d552a2f6d5649afb1a363a8b6fd611d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.693035 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-bcfc9ce3-83f4-4d09-8663-cfd9b8294744 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d552a2f6d5649afb1a363a8b6fd611d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.700892 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-bcfc9ce3-83f4-4d09-8663-cfd9b8294744 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d552a2f6d5649afb1a363a8b6fd611d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.711603 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-bcfc9ce3-83f4-4d09-8663-cfd9b8294744 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d552a2f6d5649afb1a363a8b6fd611d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.712442 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-bcfc9ce3-83f4-4d09-8663-cfd9b8294744 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d552a2f6d5649afb1a363a8b6fd611d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.733144 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:51.743744 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 868/1737] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:51 2022] DELETE /identity/v3/users/7d552a2f6d5649afb1a363a8b6fd611d => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:36:51.746273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64964fb-ec69-4063-b6de-f509c1bfc36b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:51.746931 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64964fb-ec69-4063-b6de-f509c1bfc36b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:51.747483 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64964fb-ec69-4063-b6de-f509c1bfc36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:51.756349 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995d52f-e183-4640-80f4-7f83a1eb9cb8 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:51.772994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b64964fb-ec69-4063-b6de-f509c1bfc36b None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:51.774390 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b64964fb-ec69-4063-b6de-f509c1bfc36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:51.827441 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb2c84-9b06-4370-9740-e65f882ad078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:51.828017 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb2c84-9b06-4370-9740-e65f882ad078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:51.828329 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb2c84-9b06-4370-9740-e65f882ad078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:51.828603 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb2c84-9b06-4370-9740-e65f882ad078 admin admin] PATH_INFO: `/v3/projects/b0f76b7251e54367a19e8a0a04699181` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:51.830018 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb2c84-9b06-4370-9740-e65f882ad078 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0f76b7251e54367a19e8a0a04699181)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:51.832453 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb2c84-9b06-4370-9740-e65f882ad078 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:51.851315 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 869/1738] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:51.861845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64964fb-ec69-4063-b6de-f509c1bfc36b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:51.863710 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b64964fb-ec69-4063-b6de-f509c1bfc36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:51.892498 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5cdb2c84-9b06-4370-9740-e65f882ad078 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0f76b7251e54367a19e8a0a04699181', 'request_id': 'req-5cdb2c84-9b06-4370-9740-e65f882ad078'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.900238 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5cdb2c84-9b06-4370-9740-e65f882ad078 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0f76b7251e54367a19e8a0a04699181', 'request_id': 'req-5cdb2c84-9b06-4370-9740-e65f882ad078'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.910423 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5cdb2c84-9b06-4370-9740-e65f882ad078 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0f76b7251e54367a19e8a0a04699181', 'request_id': 'req-5cdb2c84-9b06-4370-9740-e65f882ad078'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.922174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5cdb2c84-9b06-4370-9740-e65f882ad078 admin admin] The token cache is being invalidate because project b0f76b7251e54367a19e8a0a04699181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:51.922606 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5cdb2c84-9b06-4370-9740-e65f882ad078 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.923359 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5cdb2c84-9b06-4370-9740-e65f882ad078 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:51.925725 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 870/1739] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:51 2022] DELETE /identity/v3/projects/b0f76b7251e54367a19e8a0a04699181 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:51.926051 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64964fb-ec69-4063-b6de-f509c1bfc36b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:51.926854 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb2c84-9b06-4370-9740-e65f882ad078 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:51.929563 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-5cdb2c84-9b06-4370-9740-e65f882ad078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:51.961221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:51.977158 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f3f93-a665-4b23-a1b6-10fbbc49de25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:51.977773 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f3f93-a665-4b23-a1b6-10fbbc49de25 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:51.978332 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f3f93-a665-4b23-a1b6-10fbbc49de25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:51.978332 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f3f93-a665-4b23-a1b6-10fbbc49de25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:51.980074 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-597f3f93-a665-4b23-a1b6-10fbbc49de25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:51.995970 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb2c84-9b06-4370-9740-e65f882ad078 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:52.000674 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dce3fe-1ef8-4ff1-a0fe-de554f179787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:52.001343 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dce3fe-1ef8-4ff1-a0fe-de554f179787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:52.001630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dce3fe-1ef8-4ff1-a0fe-de554f179787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:52.001926 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dce3fe-1ef8-4ff1-a0fe-de554f179787 admin admin] PATH_INFO: `/v3/projects/97ad6ebdac2d4e13871d268804f3340f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:52.003673 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dce3fe-1ef8-4ff1-a0fe-de554f179787 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97ad6ebdac2d4e13871d268804f3340f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:52.005643 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dce3fe-1ef8-4ff1-a0fe-de554f179787 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:52.046793 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f3f93-a665-4b23-a1b6-10fbbc49de25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:52.048772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f3f93-a665-4b23-a1b6-10fbbc49de25 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:52.055854 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-d3dce3fe-1ef8-4ff1-a0fe-de554f179787 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97ad6ebdac2d4e13871d268804f3340f', 'request_id': 'req-d3dce3fe-1ef8-4ff1-a0fe-de554f179787'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.059739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 870/1740] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:52.060806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f3f93-a665-4b23-a1b6-10fbbc49de25 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:52.062690 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-597f3f93-a665-4b23-a1b6-10fbbc49de25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:52.063416 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-d3dce3fe-1ef8-4ff1-a0fe-de554f179787 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97ad6ebdac2d4e13871d268804f3340f', 'request_id': 'req-d3dce3fe-1ef8-4ff1-a0fe-de554f179787'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.072951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:52.075849 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-d3dce3fe-1ef8-4ff1-a0fe-de554f179787 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97ad6ebdac2d4e13871d268804f3340f', 'request_id': 'req-d3dce3fe-1ef8-4ff1-a0fe-de554f179787'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.079331 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f3f93-a665-4b23-a1b6-10fbbc49de25 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:52.093291 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-d3dce3fe-1ef8-4ff1-a0fe-de554f179787 admin admin] The token cache is being invalidate because project 97ad6ebdac2d4e13871d268804f3340f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:52.094068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-d3dce3fe-1ef8-4ff1-a0fe-de554f179787 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.095068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-d3dce3fe-1ef8-4ff1-a0fe-de554f179787 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.097108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 871/1741] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:51 2022] DELETE /identity/v3/projects/97ad6ebdac2d4e13871d268804f3340f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:52.101678 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c42ef3-0fc9-44ff-a36a-7144ce4e9a8c None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:52.102070 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c42ef3-0fc9-44ff-a36a-7144ce4e9a8c None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:52.102542 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c42ef3-0fc9-44ff-a36a-7144ce4e9a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:52.125869 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-b2c42ef3-0fc9-44ff-a36a-7144ce4e9a8c None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:52.127834 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b2c42ef3-0fc9-44ff-a36a-7144ce4e9a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:52.154853 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a8961-abca-48cd-90eb-b6984e9d89d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:52.155766 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a8961-abca-48cd-90eb-b6984e9d89d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:52.155970 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a8961-abca-48cd-90eb-b6984e9d89d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:52.156331 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a8961-abca-48cd-90eb-b6984e9d89d0 admin admin] PATH_INFO: `/v3/users/ba91c52ff3e94bc0b98b19c8fe18f349` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:52.158649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a8961-abca-48cd-90eb-b6984e9d89d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba91c52ff3e94bc0b98b19c8fe18f349)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:52.161537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a8961-abca-48cd-90eb-b6984e9d89d0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:52.208433 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 872/1742] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:36:52.216829 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a20e94-2df3-4d06-aa14-e36182d73357 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:52.216829 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a20e94-2df3-4d06-aa14-e36182d73357 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:52.217295 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a20e94-2df3-4d06-aa14-e36182d73357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:52.228552 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b76a8961-abca-48cd-90eb-b6984e9d89d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba91c52ff3e94bc0b98b19c8fe18f349'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.229342 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b76a8961-abca-48cd-90eb-b6984e9d89d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba91c52ff3e94bc0b98b19c8fe18f349'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.238494 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b76a8961-abca-48cd-90eb-b6984e9d89d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba91c52ff3e94bc0b98b19c8fe18f349'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.241053 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-58a20e94-2df3-4d06-aa14-e36182d73357 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:52.242565 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-58a20e94-2df3-4d06-aa14-e36182d73357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:52.248958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b76a8961-abca-48cd-90eb-b6984e9d89d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba91c52ff3e94bc0b98b19c8fe18f349'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.249730 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b76a8961-abca-48cd-90eb-b6984e9d89d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba91c52ff3e94bc0b98b19c8fe18f349'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.261735 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 873/1743] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:52.266476 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a20e94-2df3-4d06-aa14-e36182d73357 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:52.271828 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 871/1744] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:52 2022] DELETE /identity/v3/users/ba91c52ff3e94bc0b98b19c8fe18f349 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:52.276225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a20e94-2df3-4d06-aa14-e36182d73357 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:52.334478 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:52.349247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454483be-c569-4195-b833-5673faab8580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:52.350363 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454483be-c569-4195-b833-5673faab8580 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:52.350678 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454483be-c569-4195-b833-5673faab8580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:52.352262 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454483be-c569-4195-b833-5673faab8580 admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:52.354582 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454483be-c569-4195-b833-5673faab8580 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:36:52.355214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454483be-c569-4195-b833-5673faab8580 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:52.359399 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454483be-c569-4195-b833-5673faab8580 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:52.379188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 874/1745] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:36:52.383966 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a8961-abca-48cd-90eb-b6984e9d89d0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:52.399152 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a8961-abca-48cd-90eb-b6984e9d89d0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:52.408056 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e716d-fa62-475b-8578-ebeb63cb38a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:52.409270 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e716d-fa62-475b-8578-ebeb63cb38a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:52.409555 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e716d-fa62-475b-8578-ebeb63cb38a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:52.409880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e716d-fa62-475b-8578-ebeb63cb38a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:52.410554 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e716d-fa62-475b-8578-ebeb63cb38a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:52.414623 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e716d-fa62-475b-8578-ebeb63cb38a5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:52.462094 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454483be-c569-4195-b833-5673faab8580 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:52.487884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 872/1746] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:52 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:36:52.493437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454483be-c569-4195-b833-5673faab8580 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:52.516024 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e716d-fa62-475b-8578-ebeb63cb38a5 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:52.518599 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d14e716d-fa62-475b-8578-ebeb63cb38a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:52.585466 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12c846-0a20-4224-9d3f-f9ecf7159490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:52.586482 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12c846-0a20-4224-9d3f-f9ecf7159490 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:52.586877 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12c846-0a20-4224-9d3f-f9ecf7159490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:52.587790 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12c846-0a20-4224-9d3f-f9ecf7159490 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:52.588391 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12c846-0a20-4224-9d3f-f9ecf7159490 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:52.590421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12c846-0a20-4224-9d3f-f9ecf7159490 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:52.608008 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e716d-fa62-475b-8578-ebeb63cb38a5 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:52.613511 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69c8a2-603b-4682-a316-9ad64589ca58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:52.614321 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69c8a2-603b-4682-a316-9ad64589ca58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:52.614639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69c8a2-603b-4682-a316-9ad64589ca58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:52.614914 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69c8a2-603b-4682-a316-9ad64589ca58 admin admin] PATH_INFO: `/v3/projects/1b9b8008f9a74bf19c15360e9bf26bb9` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:52.616066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69c8a2-603b-4682-a316-9ad64589ca58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b9b8008f9a74bf19c15360e9bf26bb9)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:52.618054 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69c8a2-603b-4682-a316-9ad64589ca58 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:52.618848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 875/1747] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:52 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:52.621846 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:52.624137 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12c846-0a20-4224-9d3f-f9ecf7159490 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:52.636935 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12c846-0a20-4224-9d3f-f9ecf7159490 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:52.644053 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f1984-bedc-47dd-83a3-bfac992994d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:52.644396 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f1984-bedc-47dd-83a3-bfac992994d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:52.644624 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f1984-bedc-47dd-83a3-bfac992994d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:52.644932 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f1984-bedc-47dd-83a3-bfac992994d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:52.645596 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f1984-bedc-47dd-83a3-bfac992994d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:52.647306 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f1984-bedc-47dd-83a3-bfac992994d9 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:52.665061 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 876/1748] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:52.675137 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f1984-bedc-47dd-83a3-bfac992994d9 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:52.695706 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f1984-bedc-47dd-83a3-bfac992994d9 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:52.710843 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6b69c8a2-603b-4682-a316-9ad64589ca58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b9b8008f9a74bf19c15360e9bf26bb9', 'request_id': 'req-6b69c8a2-603b-4682-a316-9ad64589ca58'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.718020 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6b69c8a2-603b-4682-a316-9ad64589ca58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b9b8008f9a74bf19c15360e9bf26bb9', 'request_id': 'req-6b69c8a2-603b-4682-a316-9ad64589ca58'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.729674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6b69c8a2-603b-4682-a316-9ad64589ca58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b9b8008f9a74bf19c15360e9bf26bb9', 'request_id': 'req-6b69c8a2-603b-4682-a316-9ad64589ca58'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.740791 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6b69c8a2-603b-4682-a316-9ad64589ca58 admin admin] The token cache is being invalidate because project 1b9b8008f9a74bf19c15360e9bf26bb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:52.741127 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6b69c8a2-603b-4682-a316-9ad64589ca58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.741876 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-6b69c8a2-603b-4682-a316-9ad64589ca58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.743413 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 873/1749] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:52 2022] DELETE /identity/v3/projects/1b9b8008f9a74bf19c15360e9bf26bb9 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:52.747175 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69c8a2-603b-4682-a316-9ad64589ca58 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:52.748463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6b69c8a2-603b-4682-a316-9ad64589ca58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:52.763776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f2a0f-e6aa-4233-aaae-6b0cf2b4d7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:52.764480 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f2a0f-e6aa-4233-aaae-6b0cf2b4d7fc admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:52.764769 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f2a0f-e6aa-4233-aaae-6b0cf2b4d7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:52.765094 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f2a0f-e6aa-4233-aaae-6b0cf2b4d7fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:52.765737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98f2a0f-e6aa-4233-aaae-6b0cf2b4d7fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:52.768209 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98f2a0f-e6aa-4233-aaae-6b0cf2b4d7fc admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:52.778753 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 877/1750] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:36:52.779759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f2a0f-e6aa-4233-aaae-6b0cf2b4d7fc admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:52.781505 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-f98f2a0f-e6aa-4233-aaae-6b0cf2b4d7fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:52.795356 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f2a0f-e6aa-4233-aaae-6b0cf2b4d7fc admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:52.802009 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2eaf2f-79aa-405e-9840-5d05e024790c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:52.802190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2eaf2f-79aa-405e-9840-5d05e024790c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:52.802592 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2eaf2f-79aa-405e-9840-5d05e024790c admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:52.802870 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2eaf2f-79aa-405e-9840-5d05e024790c admin admin] PATH_INFO: `/v3/users/70f0fd9b81bf4bc5b52ee05a63ab4a1d` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:52.804118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2eaf2f-79aa-405e-9840-5d05e024790c admin admin] RBAC: Authorizing `identity:delete_user(user_id=70f0fd9b81bf4bc5b52ee05a63ab4a1d)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:52.806272 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2eaf2f-79aa-405e-9840-5d05e024790c admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:52.820569 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69c8a2-603b-4682-a316-9ad64589ca58 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:52.829113 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55f182-9fc6-482d-9516-a9f41bc4efcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:52.830685 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55f182-9fc6-482d-9516-a9f41bc4efcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:52.831014 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55f182-9fc6-482d-9516-a9f41bc4efcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:52.831502 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55f182-9fc6-482d-9516-a9f41bc4efcf admin admin] PATH_INFO: `/v3/projects/72f8dda61d3c4561ac5af3f496a39ad2` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:52.833657 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55f182-9fc6-482d-9516-a9f41bc4efcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=72f8dda61d3c4561ac5af3f496a39ad2)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:52.835702 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55f182-9fc6-482d-9516-a9f41bc4efcf admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:52.862139 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5f2eaf2f-79aa-405e-9840-5d05e024790c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70f0fd9b81bf4bc5b52ee05a63ab4a1d'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.863327 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5f2eaf2f-79aa-405e-9840-5d05e024790c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70f0fd9b81bf4bc5b52ee05a63ab4a1d'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.872630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5f2eaf2f-79aa-405e-9840-5d05e024790c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70f0fd9b81bf4bc5b52ee05a63ab4a1d'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.873315 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5f2eaf2f-79aa-405e-9840-5d05e024790c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70f0fd9b81bf4bc5b52ee05a63ab4a1d'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.875480 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:52.883111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5f2eaf2f-79aa-405e-9840-5d05e024790c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70f0fd9b81bf4bc5b52ee05a63ab4a1d'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.893108 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fa55f182-9fc6-482d-9516-a9f41bc4efcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72f8dda61d3c4561ac5af3f496a39ad2', 'request_id': 'req-fa55f182-9fc6-482d-9516-a9f41bc4efcf'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.901632 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fa55f182-9fc6-482d-9516-a9f41bc4efcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72f8dda61d3c4561ac5af3f496a39ad2', 'request_id': 'req-fa55f182-9fc6-482d-9516-a9f41bc4efcf'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.908046 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 878/1751] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:52 2022] DELETE /identity/v3/users/70f0fd9b81bf4bc5b52ee05a63ab4a1d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:52.908046 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2eaf2f-79aa-405e-9840-5d05e024790c admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:52.908046 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-5f2eaf2f-79aa-405e-9840-5d05e024790c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:52.914967 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fa55f182-9fc6-482d-9516-a9f41bc4efcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72f8dda61d3c4561ac5af3f496a39ad2', 'request_id': 'req-fa55f182-9fc6-482d-9516-a9f41bc4efcf'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.935093 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fa55f182-9fc6-482d-9516-a9f41bc4efcf admin admin] The token cache is being invalidate because project 72f8dda61d3c4561ac5af3f496a39ad2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:52.935439 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fa55f182-9fc6-482d-9516-a9f41bc4efcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.936752 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fa55f182-9fc6-482d-9516-a9f41bc4efcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:52.942841 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 874/1752] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:52 2022] DELETE /identity/v3/projects/72f8dda61d3c4561ac5af3f496a39ad2 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:52.944730 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a21c4f-1dd3-4ff3-89c1-616dba3c740c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:52.945420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a21c4f-1dd3-4ff3-89c1-616dba3c740c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:52.945420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a21c4f-1dd3-4ff3-89c1-616dba3c740c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:52.979597 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-17a21c4f-1dd3-4ff3-89c1-616dba3c740c None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:52.981945 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-17a21c4f-1dd3-4ff3-89c1-616dba3c740c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:53.008405 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:53.019860 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2eaf2f-79aa-405e-9840-5d05e024790c admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:53.028396 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c95dba-fab6-4619-9124-17b45eff3916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:53.030262 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c95dba-fab6-4619-9124-17b45eff3916 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:53.030973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c95dba-fab6-4619-9124-17b45eff3916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:53.031679 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c95dba-fab6-4619-9124-17b45eff3916 admin admin] PATH_INFO: `/v3/projects/3e50454e1f9946d9aedcd21f8ebd58b3/users/3d633b9b875f47fb841091cee5f0a1db/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:53.054754 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c95dba-fab6-4619-9124-17b45eff3916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e50454e1f9946d9aedcd21f8ebd58b3, user_id=3d633b9b875f47fb841091cee5f0a1db, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:53.059498 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c95dba-fab6-4619-9124-17b45eff3916 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:53.082623 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 875/1753] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:53.091374 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3f3f8-34d9-474f-bf1a-a455bf8a540b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:53.092022 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3f3f8-34d9-474f-bf1a-a455bf8a540b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:53.092436 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3f3f8-34d9-474f-bf1a-a455bf8a540b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:53.097292 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 879/1754] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:52 2022] PUT /identity/v3/projects/3e50454e1f9946d9aedcd21f8ebd58b3/users/3d633b9b875f47fb841091cee5f0a1db/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:53.099410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981a307-84c6-4864-b943-329f2b5e207c None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:53.100027 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981a307-84c6-4864-b943-329f2b5e207c None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:53.100479 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981a307-84c6-4864-b943-329f2b5e207c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:53.131350 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-82e3f3f8-34d9-474f-bf1a-a455bf8a540b None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:53.132839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-82e3f3f8-34d9-474f-bf1a-a455bf8a540b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:53.141596 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-5981a307-84c6-4864-b943-329f2b5e207c None None] MFA Rules not processed for user `3d633b9b875f47fb841091cee5f0a1db`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:53.143162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-5981a307-84c6-4864-b943-329f2b5e207c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:53.215977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 876/1755] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:36:53.220163 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3f3f8-34d9-474f-bf1a-a455bf8a540b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:53.231129 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3f3f8-34d9-474f-bf1a-a455bf8a540b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:53.231905 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 880/1756] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:53.237696 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981a307-84c6-4864-b943-329f2b5e207c None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:53.238381 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26906148-3687-4e71-beab-819311b20dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:53.239159 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26906148-3687-4e71-beab-819311b20dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:53.239345 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26906148-3687-4e71-beab-819311b20dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:53.239634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26906148-3687-4e71-beab-819311b20dc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:53.240330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-5981a307-84c6-4864-b943-329f2b5e207c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:53.240438 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26906148-3687-4e71-beab-819311b20dc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:36:53.241469 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26906148-3687-4e71-beab-819311b20dc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:53.244513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26906148-3687-4e71-beab-819311b20dc4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:53.258733 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981a307-84c6-4864-b943-329f2b5e207c None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:53.262913 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 877/1757] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:36:53.267186 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26906148-3687-4e71-beab-819311b20dc4 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:53.278441 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26906148-3687-4e71-beab-819311b20dc4 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:53.282745 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816afef-126a-46fe-9e2a-00982b9b5e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:53.283691 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816afef-126a-46fe-9e2a-00982b9b5e20 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:53.283980 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816afef-126a-46fe-9e2a-00982b9b5e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:53.284296 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816afef-126a-46fe-9e2a-00982b9b5e20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:53.285703 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816afef-126a-46fe-9e2a-00982b9b5e20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:53.287671 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816afef-126a-46fe-9e2a-00982b9b5e20 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:53.296010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a105f3-5fa5-49d6-a0f6-141825c21a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:53.296975 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a105f3-5fa5-49d6-a0f6-141825c21a46 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:53.297654 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a105f3-5fa5-49d6-a0f6-141825c21a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:53.298591 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a105f3-5fa5-49d6-a0f6-141825c21a46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:53.301696 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a105f3-5fa5-49d6-a0f6-141825c21a46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:53.306991 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a105f3-5fa5-49d6-a0f6-141825c21a46 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:53.340371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 881/1758] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:53 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:53.341561 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816afef-126a-46fe-9e2a-00982b9b5e20 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:53.353527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 878/1759] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:53 2022] POST /identity/v3/projects => generated 359 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:53.354717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a105f3-5fa5-49d6-a0f6-141825c21a46 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:53.378284 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:53.434883 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816afef-126a-46fe-9e2a-00982b9b5e20 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:53.440328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160aa6fe-88e4-4179-bea6-79b9354ac339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:53.441683 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160aa6fe-88e4-4179-bea6-79b9354ac339 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:53.441971 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160aa6fe-88e4-4179-bea6-79b9354ac339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:53.442545 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160aa6fe-88e4-4179-bea6-79b9354ac339 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:53.443211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160aa6fe-88e4-4179-bea6-79b9354ac339 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:53.444559 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a105f3-5fa5-49d6-a0f6-141825c21a46 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:53.446047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160aa6fe-88e4-4179-bea6-79b9354ac339 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:53.450288 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764bb497-3400-4b68-b388-9096ddc516de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:53.450971 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764bb497-3400-4b68-b388-9096ddc516de admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:53.451503 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764bb497-3400-4b68-b388-9096ddc516de admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:53.451797 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764bb497-3400-4b68-b388-9096ddc516de admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:53.453195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764bb497-3400-4b68-b388-9096ddc516de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:53.456837 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764bb497-3400-4b68-b388-9096ddc516de admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:53.480526 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 882/1760] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:53 2022] POST /identity/v3/users => generated 292 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:53.485240 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160aa6fe-88e4-4179-bea6-79b9354ac339 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:53.497773 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160aa6fe-88e4-4179-bea6-79b9354ac339 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:53.505216 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83637da-c31a-4af8-87c0-ef40dcb2468e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:53.505940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83637da-c31a-4af8-87c0-ef40dcb2468e admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:53.506226 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83637da-c31a-4af8-87c0-ef40dcb2468e admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:53.506508 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83637da-c31a-4af8-87c0-ef40dcb2468e admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:53.507332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83637da-c31a-4af8-87c0-ef40dcb2468e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:53.509448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 879/1761] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:53 2022] POST /identity/v3/users => generated 280 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:36:53.514209 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83637da-c31a-4af8-87c0-ef40dcb2468e admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:53.516542 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764bb497-3400-4b68-b388-9096ddc516de admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:53.530225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 883/1762] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:53.534731 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83637da-c31a-4af8-87c0-ef40dcb2468e admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:53.537294 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-f83637da-c31a-4af8-87c0-ef40dcb2468e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:53.541782 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764bb497-3400-4b68-b388-9096ddc516de admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:53.546918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b78e7f-a01f-4120-9412-b89881bc4e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:53.547668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b78e7f-a01f-4120-9412-b89881bc4e19 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:53.548046 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b78e7f-a01f-4120-9412-b89881bc4e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:53.548417 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b78e7f-a01f-4120-9412-b89881bc4e19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:53.549267 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b78e7f-a01f-4120-9412-b89881bc4e19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:53.551316 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b78e7f-a01f-4120-9412-b89881bc4e19 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:53.558257 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83637da-c31a-4af8-87c0-ef40dcb2468e admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:53.563350 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 880/1763] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:53.567282 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b35e8-860e-44b2-9b60-c89712c2b5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:53.567441 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b78e7f-a01f-4120-9412-b89881bc4e19 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:53.567538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b35e8-860e-44b2-9b60-c89712c2b5d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:53.567538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b35e8-860e-44b2-9b60-c89712c2b5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:53.567538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b35e8-860e-44b2-9b60-c89712c2b5d7 admin admin] PATH_INFO: `/v3/users/aafdb707212744ca92e317cda05044db` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:53.568610 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b35e8-860e-44b2-9b60-c89712c2b5d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aafdb707212744ca92e317cda05044db)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:53.571130 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b35e8-860e-44b2-9b60-c89712c2b5d7 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:53.583249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b78e7f-a01f-4120-9412-b89881bc4e19 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:53.588588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17289b08-0034-465a-9a0f-cb7350e1d916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:53.589356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17289b08-0034-465a-9a0f-cb7350e1d916 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:53.589748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17289b08-0034-465a-9a0f-cb7350e1d916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:53.590120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17289b08-0034-465a-9a0f-cb7350e1d916 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:53.590952 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17289b08-0034-465a-9a0f-cb7350e1d916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:53.592593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17289b08-0034-465a-9a0f-cb7350e1d916 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:53.594553 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:53.613249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 881/1764] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:53.614262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17289b08-0034-465a-9a0f-cb7350e1d916 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:53.634569 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17289b08-0034-465a-9a0f-cb7350e1d916 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:53.644910 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698fe180-1cf6-4ba8-a2dc-3d38fe331932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:53.645480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698fe180-1cf6-4ba8-a2dc-3d38fe331932 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:53.645839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698fe180-1cf6-4ba8-a2dc-3d38fe331932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:53.647929 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698fe180-1cf6-4ba8-a2dc-3d38fe331932 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:53.648070 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698fe180-1cf6-4ba8-a2dc-3d38fe331932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:53.650748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698fe180-1cf6-4ba8-a2dc-3d38fe331932 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:53.668074 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 882/1765] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:53.669867 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698fe180-1cf6-4ba8-a2dc-3d38fe331932 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:53.672149 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-698fe180-1cf6-4ba8-a2dc-3d38fe331932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:53.687319 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-370b35e8-860e-44b2-9b60-c89712c2b5d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aafdb707212744ca92e317cda05044db'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:53.688484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698fe180-1cf6-4ba8-a2dc-3d38fe331932 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:53.691657 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-370b35e8-860e-44b2-9b60-c89712c2b5d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aafdb707212744ca92e317cda05044db'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:53.718152 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f50fc-a13a-4230-a16c-a195f5f7b6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:53.719453 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f50fc-a13a-4230-a16c-a195f5f7b6cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:53.719965 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f50fc-a13a-4230-a16c-a195f5f7b6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:53.720248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f50fc-a13a-4230-a16c-a195f5f7b6cf admin admin] PATH_INFO: `/v3/projects/5b5bccc178424568a83b8eedc8728948` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:53.722104 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f50fc-a13a-4230-a16c-a195f5f7b6cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b5bccc178424568a83b8eedc8728948)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:53.724693 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f50fc-a13a-4230-a16c-a195f5f7b6cf admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:53.726173 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-370b35e8-860e-44b2-9b60-c89712c2b5d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aafdb707212744ca92e317cda05044db'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:53.728146 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-370b35e8-860e-44b2-9b60-c89712c2b5d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aafdb707212744ca92e317cda05044db'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:53.745245 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-370b35e8-860e-44b2-9b60-c89712c2b5d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aafdb707212744ca92e317cda05044db'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:53.780921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 884/1766] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:53 2022] DELETE /identity/v3/users/aafdb707212744ca92e317cda05044db => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:53.780921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b35e8-860e-44b2-9b60-c89712c2b5d7 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:53.782653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-370b35e8-860e-44b2-9b60-c89712c2b5d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:53.806989 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7f0f50fc-a13a-4230-a16c-a195f5f7b6cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b5bccc178424568a83b8eedc8728948', 'request_id': 'req-7f0f50fc-a13a-4230-a16c-a195f5f7b6cf'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:53.817975 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7f0f50fc-a13a-4230-a16c-a195f5f7b6cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b5bccc178424568a83b8eedc8728948', 'request_id': 'req-7f0f50fc-a13a-4230-a16c-a195f5f7b6cf'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:53.836250 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7f0f50fc-a13a-4230-a16c-a195f5f7b6cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b5bccc178424568a83b8eedc8728948', 'request_id': 'req-7f0f50fc-a13a-4230-a16c-a195f5f7b6cf'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:53.850786 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7f0f50fc-a13a-4230-a16c-a195f5f7b6cf admin admin] The token cache is being invalidate because project 5b5bccc178424568a83b8eedc8728948 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:53.850985 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7f0f50fc-a13a-4230-a16c-a195f5f7b6cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:53.851751 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-7f0f50fc-a13a-4230-a16c-a195f5f7b6cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:53.853758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 883/1767] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:53 2022] DELETE /identity/v3/projects/5b5bccc178424568a83b8eedc8728948 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:53.854570 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f50fc-a13a-4230-a16c-a195f5f7b6cf admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:53.856168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7f0f50fc-a13a-4230-a16c-a195f5f7b6cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:53.870416 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b35e8-860e-44b2-9b60-c89712c2b5d7 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:53.889141 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:53.923400 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f50fc-a13a-4230-a16c-a195f5f7b6cf admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:53.928670 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e7266-e259-47e3-adeb-9eb458474707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:53.929525 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e7266-e259-47e3-adeb-9eb458474707 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:53.929811 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e7266-e259-47e3-adeb-9eb458474707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:53.930051 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e7266-e259-47e3-adeb-9eb458474707 admin admin] PATH_INFO: `/v3/projects/bb29943bbbea4d81af7f4f932a6cc18d/users/72e4a0fcd14b4253acca69803a7db047/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:53.933765 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8331b-e4e9-4e6a-976f-077b847c44df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:53.934441 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8331b-e4e9-4e6a-976f-077b847c44df admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:53.934722 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8331b-e4e9-4e6a-976f-077b847c44df admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:53.934995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8331b-e4e9-4e6a-976f-077b847c44df admin admin] PATH_INFO: `/v3/projects/f0ef9819949e4b1d938213eee8db72de/users/68a7a9ac0445481186518adb11cdbf6c/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:53.946905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177e7266-e259-47e3-adeb-9eb458474707 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb29943bbbea4d81af7f4f932a6cc18d, user_id=72e4a0fcd14b4253acca69803a7db047, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:53.950525 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177e7266-e259-47e3-adeb-9eb458474707 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:53.951572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c8331b-e4e9-4e6a-976f-077b847c44df admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0ef9819949e4b1d938213eee8db72de, user_id=68a7a9ac0445481186518adb11cdbf6c, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:53.954548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c8331b-e4e9-4e6a-976f-077b847c44df admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:53.965400 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 884/1768] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:53 2022] PUT /identity/v3/projects/bb29943bbbea4d81af7f4f932a6cc18d/users/72e4a0fcd14b4253acca69803a7db047/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:53.975641 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4110a-8867-4244-970f-50927c98c3c1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:53.975641 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4110a-8867-4244-970f-50927c98c3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:53.975641 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4110a-8867-4244-970f-50927c98c3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:53.975879 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 885/1769] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:53 2022] PUT /identity/v3/projects/f0ef9819949e4b1d938213eee8db72de/users/68a7a9ac0445481186518adb11cdbf6c/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:53.975879 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8331b-e4e9-4e6a-976f-077b847c44df admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:53.982851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-26c8331b-e4e9-4e6a-976f-077b847c44df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:53.996811 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7bf4110a-8867-4244-970f-50927c98c3c1 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:53.998072 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7bf4110a-8867-4244-970f-50927c98c3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:54.049840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:54.060523 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8331b-e4e9-4e6a-976f-077b847c44df admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:54.067612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34506ad8-39bf-41c0-88e9-e12134114718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:54.068256 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34506ad8-39bf-41c0-88e9-e12134114718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:54.068666 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34506ad8-39bf-41c0-88e9-e12134114718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:54.068988 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34506ad8-39bf-41c0-88e9-e12134114718 admin admin] PATH_INFO: `/v3/users/3c35c7cbe30c4da0818ba2ea95bfcb2b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:54.070121 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34506ad8-39bf-41c0-88e9-e12134114718 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c35c7cbe30c4da0818ba2ea95bfcb2b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:54.072544 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 885/1770] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:54.072892 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34506ad8-39bf-41c0-88e9-e12134114718 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:54.073989 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639f40d-4e0f-4a4f-89ee-299abc44a46e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:54.074280 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639f40d-4e0f-4a4f-89ee-299abc44a46e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:54.074558 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639f40d-4e0f-4a4f-89ee-299abc44a46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:54.104435 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d639f40d-4e0f-4a4f-89ee-299abc44a46e None None] MFA Rules not processed for user `72e4a0fcd14b4253acca69803a7db047`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:54.105654 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d639f40d-4e0f-4a4f-89ee-299abc44a46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:54.128657 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-34506ad8-39bf-41c0-88e9-e12134114718 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c35c7cbe30c4da0818ba2ea95bfcb2b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:54.129449 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-34506ad8-39bf-41c0-88e9-e12134114718 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c35c7cbe30c4da0818ba2ea95bfcb2b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:54.138731 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-34506ad8-39bf-41c0-88e9-e12134114718 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c35c7cbe30c4da0818ba2ea95bfcb2b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:54.139395 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-34506ad8-39bf-41c0-88e9-e12134114718 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c35c7cbe30c4da0818ba2ea95bfcb2b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:54.148895 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-34506ad8-39bf-41c0-88e9-e12134114718 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c35c7cbe30c4da0818ba2ea95bfcb2b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:54.171002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 886/1771] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:53 2022] DELETE /identity/v3/users/3c35c7cbe30c4da0818ba2ea95bfcb2b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:54.172283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75827e07-1eb3-4eb6-9fc0-687c5a427889 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:54.172569 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75827e07-1eb3-4eb6-9fc0-687c5a427889 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:54.172841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75827e07-1eb3-4eb6-9fc0-687c5a427889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:54.198545 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 886/1772] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:54 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:54.199490 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639f40d-4e0f-4a4f-89ee-299abc44a46e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:54.201050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d639f40d-4e0f-4a4f-89ee-299abc44a46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:54.223644 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-75827e07-1eb3-4eb6-9fc0-687c5a427889 None None] MFA Rules not processed for user `68a7a9ac0445481186518adb11cdbf6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:54.225124 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-75827e07-1eb3-4eb6-9fc0-687c5a427889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:54.243219 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:54.291004 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639f40d-4e0f-4a4f-89ee-299abc44a46e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:54.324190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 887/1773] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:54.324190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75827e07-1eb3-4eb6-9fc0-687c5a427889 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:54.324190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-75827e07-1eb3-4eb6-9fc0-687c5a427889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:54.345171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48129a05-ecea-40e0-a998-f40461f096b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:54.345909 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48129a05-ecea-40e0-a998-f40461f096b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:54.346214 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48129a05-ecea-40e0-a998-f40461f096b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:54.346478 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48129a05-ecea-40e0-a998-f40461f096b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:54.348504 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-48129a05-ecea-40e0-a998-f40461f096b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:54.404789 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75827e07-1eb3-4eb6-9fc0-687c5a427889 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:54.406771 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:54.410672 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542aec1c-1dea-4aad-b5f9-948f2366f2a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:54.411372 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542aec1c-1dea-4aad-b5f9-948f2366f2a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:54.411567 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542aec1c-1dea-4aad-b5f9-948f2366f2a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:54.411852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542aec1c-1dea-4aad-b5f9-948f2366f2a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:54.426349 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48129a05-ecea-40e0-a998-f40461f096b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:54.427785 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48129a05-ecea-40e0-a998-f40461f096b3 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:54.438894 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 887/1774] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:54.440322 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48129a05-ecea-40e0-a998-f40461f096b3 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:54.441803 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-48129a05-ecea-40e0-a998-f40461f096b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:54.454452 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48129a05-ecea-40e0-a998-f40461f096b3 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:54.461153 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be31363-c69b-4703-baff-6db2da0d61dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:54.461769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be31363-c69b-4703-baff-6db2da0d61dc admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:54.462387 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be31363-c69b-4703-baff-6db2da0d61dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:54.462686 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be31363-c69b-4703-baff-6db2da0d61dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:54.463329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be31363-c69b-4703-baff-6db2da0d61dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:54.466190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be31363-c69b-4703-baff-6db2da0d61dc admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:54.490567 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542aec1c-1dea-4aad-b5f9-948f2366f2a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:54.492604 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542aec1c-1dea-4aad-b5f9-948f2366f2a7 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:54.504283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 888/1775] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:54 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:54.505398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542aec1c-1dea-4aad-b5f9-948f2366f2a7 service cinder] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:54.513717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 888/1776] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:54 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:54.513945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-542aec1c-1dea-4aad-b5f9-948f2366f2a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:54.514733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be31363-c69b-4703-baff-6db2da0d61dc admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:54.541376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542aec1c-1dea-4aad-b5f9-948f2366f2a7 service cinder] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:54.632133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be31363-c69b-4703-baff-6db2da0d61dc admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:54.633016 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09966299-773b-4a50-bc88-9df666a38127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:54.634042 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09966299-773b-4a50-bc88-9df666a38127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:54.634322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09966299-773b-4a50-bc88-9df666a38127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:54.634654 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09966299-773b-4a50-bc88-9df666a38127 admin admin] PATH_INFO: `/v3/projects/eda04a1c1fe54ca1a78cb5b27cef211e` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:54.635996 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09966299-773b-4a50-bc88-9df666a38127 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eda04a1c1fe54ca1a78cb5b27cef211e)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:54.638451 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09966299-773b-4a50-bc88-9df666a38127 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:54.639006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874aae85-fd13-4f6a-bce5-078a393426a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:54.639926 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874aae85-fd13-4f6a-bce5-078a393426a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:54.640420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874aae85-fd13-4f6a-bce5-078a393426a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:54.640823 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874aae85-fd13-4f6a-bce5-078a393426a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:54.642213 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874aae85-fd13-4f6a-bce5-078a393426a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:54.646297 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874aae85-fd13-4f6a-bce5-078a393426a1 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:54.655841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:54.678132 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 889/1777] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:54 2022] POST /identity/v3/users => generated 299 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:54.679224 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874aae85-fd13-4f6a-bce5-078a393426a1 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:54.681434 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-874aae85-fd13-4f6a-bce5-078a393426a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:54.698057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874aae85-fd13-4f6a-bce5-078a393426a1 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:54.704134 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645fab08-d331-4939-97a5-35aa0f6d6f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:54.704888 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fab08-d331-4939-97a5-35aa0f6d6f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:54.705861 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fab08-d331-4939-97a5-35aa0f6d6f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:54.706095 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fab08-d331-4939-97a5-35aa0f6d6f85 admin admin] PATH_INFO: `/v3/users/72e4a0fcd14b4253acca69803a7db047` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:54.707478 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645fab08-d331-4939-97a5-35aa0f6d6f85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72e4a0fcd14b4253acca69803a7db047)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:54.711215 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645fab08-d331-4939-97a5-35aa0f6d6f85 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:54.745800 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-09966299-773b-4a50-bc88-9df666a38127 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eda04a1c1fe54ca1a78cb5b27cef211e', 'request_id': 'req-09966299-773b-4a50-bc88-9df666a38127'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:54.760933 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-09966299-773b-4a50-bc88-9df666a38127 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eda04a1c1fe54ca1a78cb5b27cef211e', 'request_id': 'req-09966299-773b-4a50-bc88-9df666a38127'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:54.772129 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-09966299-773b-4a50-bc88-9df666a38127 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eda04a1c1fe54ca1a78cb5b27cef211e', 'request_id': 'req-09966299-773b-4a50-bc88-9df666a38127'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:54.783484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-645fab08-d331-4939-97a5-35aa0f6d6f85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72e4a0fcd14b4253acca69803a7db047'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:54.784643 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-645fab08-d331-4939-97a5-35aa0f6d6f85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72e4a0fcd14b4253acca69803a7db047'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:54.795247 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-645fab08-d331-4939-97a5-35aa0f6d6f85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72e4a0fcd14b4253acca69803a7db047'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:54.810614 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-09966299-773b-4a50-bc88-9df666a38127 admin admin] The token cache is being invalidate because project eda04a1c1fe54ca1a78cb5b27cef211e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:54.810959 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-09966299-773b-4a50-bc88-9df666a38127 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:54.811595 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-09966299-773b-4a50-bc88-9df666a38127 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:54.815938 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-645fab08-d331-4939-97a5-35aa0f6d6f85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72e4a0fcd14b4253acca69803a7db047'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:54.816102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 889/1778] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:54 2022] DELETE /identity/v3/projects/eda04a1c1fe54ca1a78cb5b27cef211e => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:54.816393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09966299-773b-4a50-bc88-9df666a38127 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:54.817978 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-09966299-773b-4a50-bc88-9df666a38127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:54.821682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-645fab08-d331-4939-97a5-35aa0f6d6f85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72e4a0fcd14b4253acca69803a7db047'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:54.874860 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 890/1779] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:54 2022] DELETE /identity/v3/users/72e4a0fcd14b4253acca69803a7db047 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:54.878552 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320256d-7e2f-458d-a50a-670953942a13 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:54.879142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320256d-7e2f-458d-a50a-670953942a13 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:54.879636 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320256d-7e2f-458d-a50a-670953942a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:54.899656 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a320256d-7e2f-458d-a50a-670953942a13 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:54.900841 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a320256d-7e2f-458d-a50a-670953942a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:54.904070 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09966299-773b-4a50-bc88-9df666a38127 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:54.923243 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:54.962746 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd4fae-98eb-404c-82de-d176be43f101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:54.963362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd4fae-98eb-404c-82de-d176be43f101 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:54.963638 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd4fae-98eb-404c-82de-d176be43f101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:54.963910 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd4fae-98eb-404c-82de-d176be43f101 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:54.964544 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd4fae-98eb-404c-82de-d176be43f101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:54.965426 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 891/1780] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:54.966150 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320256d-7e2f-458d-a50a-670953942a13 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:54.966241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd4fae-98eb-404c-82de-d176be43f101 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:54.967577 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a320256d-7e2f-458d-a50a-670953942a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:54.979152 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 890/1781] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:54.979152 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74710ba-a06e-42ca-aceb-2f602c1e92a2 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:54.979152 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74710ba-a06e-42ca-aceb-2f602c1e92a2 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:54.979152 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74710ba-a06e-42ca-aceb-2f602c1e92a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:54.979872 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320256d-7e2f-458d-a50a-670953942a13 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:54.984431 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3115f-1d44-426e-a391-43699d8a25de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:54.985134 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3115f-1d44-426e-a391-43699d8a25de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:54.985414 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3115f-1d44-426e-a391-43699d8a25de admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:54.985684 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3115f-1d44-426e-a391-43699d8a25de admin admin] PATH_INFO: `/v3/projects/0b6a8574a0d34818811deb8d18a7011b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:54.987687 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3115f-1d44-426e-a391-43699d8a25de admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b6a8574a0d34818811deb8d18a7011b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:54.989570 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3115f-1d44-426e-a391-43699d8a25de admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:54.997326 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-d74710ba-a06e-42ca-aceb-2f602c1e92a2 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:54.998521 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-d74710ba-a06e-42ca-aceb-2f602c1e92a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:55.008029 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 891/1782] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:55.009252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0ccd7-76aa-4f8c-a29e-9138da2fea48 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:55.009538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0ccd7-76aa-4f8c-a29e-9138da2fea48 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:55.009810 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0ccd7-76aa-4f8c-a29e-9138da2fea48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:55.030792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-cec0ccd7-76aa-4f8c-a29e-9138da2fea48 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:55.031981 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-cec0ccd7-76aa-4f8c-a29e-9138da2fea48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:55.037848 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c2b3115f-1d44-426e-a391-43699d8a25de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b6a8574a0d34818811deb8d18a7011b', 'request_id': 'req-c2b3115f-1d44-426e-a391-43699d8a25de'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:55.044237 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c2b3115f-1d44-426e-a391-43699d8a25de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b6a8574a0d34818811deb8d18a7011b', 'request_id': 'req-c2b3115f-1d44-426e-a391-43699d8a25de'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:55.052647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c2b3115f-1d44-426e-a391-43699d8a25de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b6a8574a0d34818811deb8d18a7011b', 'request_id': 'req-c2b3115f-1d44-426e-a391-43699d8a25de'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:55.063179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c2b3115f-1d44-426e-a391-43699d8a25de admin admin] The token cache is being invalidate because project 0b6a8574a0d34818811deb8d18a7011b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:55.063318 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:55.063747 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c2b3115f-1d44-426e-a391-43699d8a25de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:55.064971 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c2b3115f-1d44-426e-a391-43699d8a25de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:55.066702 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 892/1783] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:54 2022] DELETE /identity/v3/projects/0b6a8574a0d34818811deb8d18a7011b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:55.067538 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3115f-1d44-426e-a391-43699d8a25de admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:55.069208 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c2b3115f-1d44-426e-a391-43699d8a25de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:55.097628 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 892/1784] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:55.098557 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0ccd7-76aa-4f8c-a29e-9138da2fea48 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:55.099816 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-cec0ccd7-76aa-4f8c-a29e-9138da2fea48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:55.136354 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3115f-1d44-426e-a391-43699d8a25de admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:55.141065 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982b3b7-99d4-47fe-bda9-15789c9e7bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:55.141732 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982b3b7-99d4-47fe-bda9-15789c9e7bfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:55.142009 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982b3b7-99d4-47fe-bda9-15789c9e7bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:55.142286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982b3b7-99d4-47fe-bda9-15789c9e7bfd admin admin] PATH_INFO: `/v3/projects/974af7d6480947cea2ced410115d5f1b/users/e0aee576ef3444e79bead251b40c1df9/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:55.158709 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0ccd7-76aa-4f8c-a29e-9138da2fea48 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:55.159400 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9982b3b7-99d4-47fe-bda9-15789c9e7bfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=974af7d6480947cea2ced410115d5f1b, user_id=e0aee576ef3444e79bead251b40c1df9, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:55.162084 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9982b3b7-99d4-47fe-bda9-15789c9e7bfd admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:55.162434 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22473dee-7c87-44f0-908d-a09dd94622eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:55.163045 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22473dee-7c87-44f0-908d-a09dd94622eb service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:55.163334 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22473dee-7c87-44f0-908d-a09dd94622eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:55.163602 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22473dee-7c87-44f0-908d-a09dd94622eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:55.165516 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-22473dee-7c87-44f0-908d-a09dd94622eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:55.174907 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 893/1785] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:55 2022] PUT /identity/v3/projects/974af7d6480947cea2ced410115d5f1b/users/e0aee576ef3444e79bead251b40c1df9/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:55.175992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982b3b7-99d4-47fe-bda9-15789c9e7bfd admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:55.177794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9982b3b7-99d4-47fe-bda9-15789c9e7bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:55.180653 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22473dee-7c87-44f0-908d-a09dd94622eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:55.182139 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22473dee-7c87-44f0-908d-a09dd94622eb service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:55.192393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 893/1786] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:55.193296 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22473dee-7c87-44f0-908d-a09dd94622eb service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:55.202444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22473dee-7c87-44f0-908d-a09dd94622eb service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:55.215182 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:55.266454 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982b3b7-99d4-47fe-bda9-15789c9e7bfd admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:55.273573 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5999b2-3435-411c-80e3-3063970ab502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:55.274338 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5999b2-3435-411c-80e3-3063970ab502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:55.274587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5999b2-3435-411c-80e3-3063970ab502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:55.274789 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5999b2-3435-411c-80e3-3063970ab502 admin admin] PATH_INFO: `/v3/users/3e8609e8b4da4cf28bf82792df5e5f2b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:55.276252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5999b2-3435-411c-80e3-3063970ab502 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e8609e8b4da4cf28bf82792df5e5f2b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:55.279378 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5999b2-3435-411c-80e3-3063970ab502 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:55.282461 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662705a-35bc-4c5c-838c-a164a58e089c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:55.283766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662705a-35bc-4c5c-838c-a164a58e089c admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:55.284146 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662705a-35bc-4c5c-838c-a164a58e089c admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:55.284503 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662705a-35bc-4c5c-838c-a164a58e089c admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:55.285269 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662705a-35bc-4c5c-838c-a164a58e089c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:36:55.285772 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662705a-35bc-4c5c-838c-a164a58e089c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:55.288574 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662705a-35bc-4c5c-838c-a164a58e089c admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:55.306035 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 894/1787] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:36:55.306404 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662705a-35bc-4c5c-838c-a164a58e089c admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:55.320201 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662705a-35bc-4c5c-838c-a164a58e089c admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:55.326252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9a255-f975-4e88-9de1-16d7de3ef598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:55.326865 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9a255-f975-4e88-9de1-16d7de3ef598 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:55.327145 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9a255-f975-4e88-9de1-16d7de3ef598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:55.327420 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9a255-f975-4e88-9de1-16d7de3ef598 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:55.328272 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9a255-f975-4e88-9de1-16d7de3ef598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:55.331127 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c9a255-f975-4e88-9de1-16d7de3ef598 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:55.338885 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4a5999b2-3435-411c-80e3-3063970ab502 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e8609e8b4da4cf28bf82792df5e5f2b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:55.339553 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4a5999b2-3435-411c-80e3-3063970ab502 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e8609e8b4da4cf28bf82792df5e5f2b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:55.344404 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 895/1788] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:55.345308 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9a255-f975-4e88-9de1-16d7de3ef598 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:55.347576 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-d3c9a255-f975-4e88-9de1-16d7de3ef598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:55.351762 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4a5999b2-3435-411c-80e3-3063970ab502 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e8609e8b4da4cf28bf82792df5e5f2b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:55.363343 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9a255-f975-4e88-9de1-16d7de3ef598 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:55.364598 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4a5999b2-3435-411c-80e3-3063970ab502 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e8609e8b4da4cf28bf82792df5e5f2b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:55.366031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4a5999b2-3435-411c-80e3-3063970ab502 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e8609e8b4da4cf28bf82792df5e5f2b'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:55.371594 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23baf48a-5a2f-4dd1-a842-cd973bb0ec84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:55.372290 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23baf48a-5a2f-4dd1-a842-cd973bb0ec84 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:55.372641 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23baf48a-5a2f-4dd1-a842-cd973bb0ec84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:55.373152 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23baf48a-5a2f-4dd1-a842-cd973bb0ec84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:55.373867 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23baf48a-5a2f-4dd1-a842-cd973bb0ec84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:55.375856 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23baf48a-5a2f-4dd1-a842-cd973bb0ec84 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:55.392353 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 894/1789] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:55 2022] DELETE /identity/v3/users/3e8609e8b4da4cf28bf82792df5e5f2b => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:55.392353 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5999b2-3435-411c-80e3-3063970ab502 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:55.392353 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4a5999b2-3435-411c-80e3-3063970ab502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:55.413950 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 896/1790] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:55 2022] POST /identity/v3/projects => generated 363 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:55.417833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23baf48a-5a2f-4dd1-a842-cd973bb0ec84 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:55.481321 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5999b2-3435-411c-80e3-3063970ab502 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:55.499518 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:55.526344 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23baf48a-5a2f-4dd1-a842-cd973bb0ec84 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:55.538101 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903b05c-62cf-4cab-bd7e-8c8ce564203a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:55.539082 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903b05c-62cf-4cab-bd7e-8c8ce564203a admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:55.539597 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903b05c-62cf-4cab-bd7e-8c8ce564203a admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:55.540714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903b05c-62cf-4cab-bd7e-8c8ce564203a admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:55.541927 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903b05c-62cf-4cab-bd7e-8c8ce564203a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:55.545400 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903b05c-62cf-4cab-bd7e-8c8ce564203a admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:55.576187 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 897/1791] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:55 2022] POST /identity/v3/users => generated 282 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:36:55.578135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903b05c-62cf-4cab-bd7e-8c8ce564203a admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:55.583872 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-8903b05c-62cf-4cab-bd7e-8c8ce564203a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:55.589361 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6b87a-e76b-4f46-b519-35519082132d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:55.590129 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6b87a-e76b-4f46-b519-35519082132d admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:55.590955 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6b87a-e76b-4f46-b519-35519082132d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:55.592163 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6b87a-e76b-4f46-b519-35519082132d admin admin] PATH_INFO: `/v3/projects/974af7d6480947cea2ced410115d5f1b/users/e0aee576ef3444e79bead251b40c1df9/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:55.595898 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6b87a-e76b-4f46-b519-35519082132d admin admin] RBAC: Authorizing `identity:create_grant(project_id=974af7d6480947cea2ced410115d5f1b, user_id=e0aee576ef3444e79bead251b40c1df9, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:55.597481 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903b05c-62cf-4cab-bd7e-8c8ce564203a admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:55.602268 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6b87a-e76b-4f46-b519-35519082132d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:55.603276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff3fd6-0481-4de1-9c67-d20548e703d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:55.603914 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff3fd6-0481-4de1-9c67-d20548e703d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:55.604200 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff3fd6-0481-4de1-9c67-d20548e703d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:55.604479 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff3fd6-0481-4de1-9c67-d20548e703d0 admin admin] PATH_INFO: `/v3/projects/bb29943bbbea4d81af7f4f932a6cc18d` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:55.606004 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:55.606187 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff3fd6-0481-4de1-9c67-d20548e703d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb29943bbbea4d81af7f4f932a6cc18d)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:55.608450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff3fd6-0481-4de1-9c67-d20548e703d0 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:55.624939 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 895/1792] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:55 2022] PUT /identity/v3/projects/974af7d6480947cea2ced410115d5f1b/users/e0aee576ef3444e79bead251b40c1df9/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:55.625933 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6b87a-e76b-4f46-b519-35519082132d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:55.673546 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-1bff3fd6-0481-4de1-9c67-d20548e703d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb29943bbbea4d81af7f4f932a6cc18d', 'request_id': 'req-1bff3fd6-0481-4de1-9c67-d20548e703d0'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:55.681719 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-1bff3fd6-0481-4de1-9c67-d20548e703d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb29943bbbea4d81af7f4f932a6cc18d', 'request_id': 'req-1bff3fd6-0481-4de1-9c67-d20548e703d0'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:55.691700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-1bff3fd6-0481-4de1-9c67-d20548e703d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb29943bbbea4d81af7f4f932a6cc18d', 'request_id': 'req-1bff3fd6-0481-4de1-9c67-d20548e703d0'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:55.705024 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-1bff3fd6-0481-4de1-9c67-d20548e703d0 admin admin] The token cache is being invalidate because project bb29943bbbea4d81af7f4f932a6cc18d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:55.705353 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-1bff3fd6-0481-4de1-9c67-d20548e703d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:55.706512 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-1bff3fd6-0481-4de1-9c67-d20548e703d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:55.707296 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6b87a-e76b-4f46-b519-35519082132d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:55.711617 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 898/1793] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:55 2022] DELETE /identity/v3/projects/bb29943bbbea4d81af7f4f932a6cc18d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:55.711617 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f8dc0-f7be-4f2c-8b18-1bb8cdabeba2 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:55.711617 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f8dc0-f7be-4f2c-8b18-1bb8cdabeba2 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:55.711617 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f8dc0-f7be-4f2c-8b18-1bb8cdabeba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:55.741480 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-a70f8dc0-f7be-4f2c-8b18-1bb8cdabeba2 None None] MFA Rules not processed for user `e0aee576ef3444e79bead251b40c1df9`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:55.743189 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-a70f8dc0-f7be-4f2c-8b18-1bb8cdabeba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:55.782639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ce745-3060-4538-8603-72ceca631b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:55.783542 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ce745-3060-4538-8603-72ceca631b0a admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:55.783996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ce745-3060-4538-8603-72ceca631b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:55.784271 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ce745-3060-4538-8603-72ceca631b0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:55.784957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ce745-3060-4538-8603-72ceca631b0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:55.786544 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ce745-3060-4538-8603-72ceca631b0a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:55.796317 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 896/1794] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:55.798417 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536eb28f-f907-456c-842f-c6c9e051b3a7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:55.799379 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536eb28f-f907-456c-842f-c6c9e051b3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:55.799657 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536eb28f-f907-456c-842f-c6c9e051b3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:55.819671 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-536eb28f-f907-456c-842f-c6c9e051b3a7 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:55.820949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-536eb28f-f907-456c-842f-c6c9e051b3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:55.823829 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 899/1795] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:55.824719 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f8dc0-f7be-4f2c-8b18-1bb8cdabeba2 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:55.826004 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-a70f8dc0-f7be-4f2c-8b18-1bb8cdabeba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:55.835208 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 897/1796] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:55.835208 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536eb28f-f907-456c-842f-c6c9e051b3a7 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:55.836061 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-536eb28f-f907-456c-842f-c6c9e051b3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:55.847101 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f8dc0-f7be-4f2c-8b18-1bb8cdabeba2 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:55.847552 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536eb28f-f907-456c-842f-c6c9e051b3a7 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:55.851778 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b626d7-4b81-4973-a871-42120fc51965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:55.852052 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa84bf-9cb9-4640-95dd-38ee32a8ed33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:55.852406 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b626d7-4b81-4973-a871-42120fc51965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:55.852643 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa84bf-9cb9-4640-95dd-38ee32a8ed33 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:55.852745 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b626d7-4b81-4973-a871-42120fc51965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:55.852942 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa84bf-9cb9-4640-95dd-38ee32a8ed33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:55.853050 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b626d7-4b81-4973-a871-42120fc51965 admin admin] PATH_INFO: `/v3/projects/c18f6dbe6aac4df5aab1fbd1e2837b36` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:55.853213 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa84bf-9cb9-4640-95dd-38ee32a8ed33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:55.853846 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa84bf-9cb9-4640-95dd-38ee32a8ed33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:55.854171 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b626d7-4b81-4973-a871-42120fc51965 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c18f6dbe6aac4df5aab1fbd1e2837b36)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:55.855416 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aa84bf-9cb9-4640-95dd-38ee32a8ed33 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:55.856062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b626d7-4b81-4973-a871-42120fc51965 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:55.865810 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 898/1797] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:55.870029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa84bf-9cb9-4640-95dd-38ee32a8ed33 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:55.871911 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f7aa84bf-9cb9-4640-95dd-38ee32a8ed33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:55.895580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:55.915656 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-49b626d7-4b81-4973-a871-42120fc51965 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c18f6dbe6aac4df5aab1fbd1e2837b36', 'request_id': 'req-49b626d7-4b81-4973-a871-42120fc51965'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:55.923031 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-49b626d7-4b81-4973-a871-42120fc51965 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c18f6dbe6aac4df5aab1fbd1e2837b36', 'request_id': 'req-49b626d7-4b81-4973-a871-42120fc51965'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:55.934226 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-49b626d7-4b81-4973-a871-42120fc51965 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c18f6dbe6aac4df5aab1fbd1e2837b36', 'request_id': 'req-49b626d7-4b81-4973-a871-42120fc51965'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:55.947915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-49b626d7-4b81-4973-a871-42120fc51965 admin admin] The token cache is being invalidate because project c18f6dbe6aac4df5aab1fbd1e2837b36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:55.948394 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-49b626d7-4b81-4973-a871-42120fc51965 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:55.949290 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-49b626d7-4b81-4973-a871-42120fc51965 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:55.950539 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa84bf-9cb9-4640-95dd-38ee32a8ed33 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:55.951411 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 900/1798] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:55 2022] DELETE /identity/v3/projects/c18f6dbe6aac4df5aab1fbd1e2837b36 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:55.952854 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cad12e-b043-434d-a5fa-13f1230fe6ca None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:55.953321 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cad12e-b043-434d-a5fa-13f1230fe6ca None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:55.953839 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cad12e-b043-434d-a5fa-13f1230fe6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:55.978936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-73cad12e-b043-434d-a5fa-13f1230fe6ca None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:55.980187 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-73cad12e-b043-434d-a5fa-13f1230fe6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:55.986708 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:56.007319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6c3227-5e2e-4f49-a1e8-6b4015891345 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:56.007942 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c3227-5e2e-4f49-a1e8-6b4015891345 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:56.008220 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c3227-5e2e-4f49-a1e8-6b4015891345 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:56.008492 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c3227-5e2e-4f49-a1e8-6b4015891345 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:56.010288 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cb6c3227-5e2e-4f49-a1e8-6b4015891345 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:56.059805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 901/1799] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:56.060716 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cad12e-b043-434d-a5fa-13f1230fe6ca None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:56.062027 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-73cad12e-b043-434d-a5fa-13f1230fe6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:56.074806 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cad12e-b043-434d-a5fa-13f1230fe6ca None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:56.079555 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7eff32-6f8a-4c2b-bcc8-97de6d15d752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:56.080526 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7eff32-6f8a-4c2b-bcc8-97de6d15d752 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:56.080962 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7eff32-6f8a-4c2b-bcc8-97de6d15d752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:56.081790 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7eff32-6f8a-4c2b-bcc8-97de6d15d752 admin admin] PATH_INFO: `/v3/projects/ed7726d1a1964225845ef2a25138001d/users/2675e22c174344c4bacc13d24e425d82/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:56.083627 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c3227-5e2e-4f49-a1e8-6b4015891345 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:56.085119 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6c3227-5e2e-4f49-a1e8-6b4015891345 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:56.096102 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 899/1800] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:56.097396 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908e120-87fd-48c5-ad76-7f7ec26f3a9c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:56.097682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908e120-87fd-48c5-ad76-7f7ec26f3a9c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:56.097958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908e120-87fd-48c5-ad76-7f7ec26f3a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:56.099555 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7eff32-6f8a-4c2b-bcc8-97de6d15d752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed7726d1a1964225845ef2a25138001d, user_id=2675e22c174344c4bacc13d24e425d82, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:56.106173 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7eff32-6f8a-4c2b-bcc8-97de6d15d752 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:56.125050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d908e120-87fd-48c5-ad76-7f7ec26f3a9c None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:56.133019 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 902/1801] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:56 2022] PUT /identity/v3/projects/ed7726d1a1964225845ef2a25138001d/users/2675e22c174344c4bacc13d24e425d82/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:56.133304 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d908e120-87fd-48c5-ad76-7f7ec26f3a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:56.133384 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7eff32-6f8a-4c2b-bcc8-97de6d15d752 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:56.216051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7eff32-6f8a-4c2b-bcc8-97de6d15d752 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:56.221693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3370c93-ec56-42fb-9d6e-e2ea168720e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:56.222512 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3370c93-ec56-42fb-9d6e-e2ea168720e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:56.222790 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3370c93-ec56-42fb-9d6e-e2ea168720e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:56.223203 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 900/1802] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:56.224494 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c191250-8d0c-4735-88f2-eb2ce9590845 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:56.224780 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c191250-8d0c-4735-88f2-eb2ce9590845 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:56.225084 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c191250-8d0c-4735-88f2-eb2ce9590845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:56.230157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3370c93-ec56-42fb-9d6e-e2ea168720e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:56.231024 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3370c93-ec56-42fb-9d6e-e2ea168720e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:36:56.231497 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3370c93-ec56-42fb-9d6e-e2ea168720e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:56.233561 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3370c93-ec56-42fb-9d6e-e2ea168720e1 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:56.249857 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 903/1803] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:36:56.251122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c302f2-e98f-40b5-a06c-cbea7e79d9ad None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:56.251413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c302f2-e98f-40b5-a06c-cbea7e79d9ad None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:56.251870 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c302f2-e98f-40b5-a06c-cbea7e79d9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:56.266240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8c191250-8d0c-4735-88f2-eb2ce9590845 None None] MFA Rules not processed for user `2675e22c174344c4bacc13d24e425d82`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:56.268580 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8c191250-8d0c-4735-88f2-eb2ce9590845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:56.276187 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-70c302f2-e98f-40b5-a06c-cbea7e79d9ad None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:56.277569 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-70c302f2-e98f-40b5-a06c-cbea7e79d9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:56.289047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 904/1804] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:56.289968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c302f2-e98f-40b5-a06c-cbea7e79d9ad None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:56.308805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c302f2-e98f-40b5-a06c-cbea7e79d9ad None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:56.320948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959efef5-286b-4ace-8dcb-0f0aa532a6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:56.323500 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959efef5-286b-4ace-8dcb-0f0aa532a6d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:56.323856 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959efef5-286b-4ace-8dcb-0f0aa532a6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:56.324339 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959efef5-286b-4ace-8dcb-0f0aa532a6d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:56.325682 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959efef5-286b-4ace-8dcb-0f0aa532a6d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:56.330332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959efef5-286b-4ace-8dcb-0f0aa532a6d5 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:56.365778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 901/1805] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:56.368667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 905/1806] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:56 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:56.369820 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c191250-8d0c-4735-88f2-eb2ce9590845 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:56.373714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959efef5-286b-4ace-8dcb-0f0aa532a6d5 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:56.384195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:56.384326 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c191250-8d0c-4735-88f2-eb2ce9590845 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:56.407257 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:56.475613 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae291d-46a6-4fdf-9206-20c067030ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:56.476251 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae291d-46a6-4fdf-9206-20c067030ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:56.476548 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae291d-46a6-4fdf-9206-20c067030ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:56.476796 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae291d-46a6-4fdf-9206-20c067030ec0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:56.477471 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae291d-46a6-4fdf-9206-20c067030ec0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:36:56.477918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae291d-46a6-4fdf-9206-20c067030ec0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:56.479406 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae291d-46a6-4fdf-9206-20c067030ec0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:56.508405 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 902/1807] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:36:56.513182 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae291d-46a6-4fdf-9206-20c067030ec0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:56.515465 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f6ae291d-46a6-4fdf-9206-20c067030ec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:56.517176 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959efef5-286b-4ace-8dcb-0f0aa532a6d5 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:56.523316 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73f903-56ec-4374-a322-8e4802ebe44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:56.524568 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73f903-56ec-4374-a322-8e4802ebe44b admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:56.524568 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73f903-56ec-4374-a322-8e4802ebe44b admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:56.524964 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73f903-56ec-4374-a322-8e4802ebe44b admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:56.525894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de73f903-56ec-4374-a322-8e4802ebe44b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:56.528784 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de73f903-56ec-4374-a322-8e4802ebe44b admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:56.558444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 906/1808] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:56 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:56.558444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73f903-56ec-4374-a322-8e4802ebe44b admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:56.567158 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73f903-56ec-4374-a322-8e4802ebe44b admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:56.571939 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5edf5-c6e0-4c98-bb74-614e54422f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:56.572579 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5edf5-c6e0-4c98-bb74-614e54422f08 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:56.572902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5edf5-c6e0-4c98-bb74-614e54422f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:56.573186 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5edf5-c6e0-4c98-bb74-614e54422f08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:56.573819 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b5edf5-c6e0-4c98-bb74-614e54422f08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:56.575931 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b5edf5-c6e0-4c98-bb74-614e54422f08 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:56.576674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae291d-46a6-4fdf-9206-20c067030ec0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:56.580541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9bd5c-748f-4013-972c-564ca81f81a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:56.581073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9bd5c-748f-4013-972c-564ca81f81a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:56.581419 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9bd5c-748f-4013-972c-564ca81f81a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:56.581614 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9bd5c-748f-4013-972c-564ca81f81a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:56.604765 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 907/1809] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:56 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:56.605877 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5edf5-c6e0-4c98-bb74-614e54422f08 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:56.643284 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca9bd5c-748f-4013-972c-564ca81f81a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:56.644727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca9bd5c-748f-4013-972c-564ca81f81a7 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:56.653741 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 903/1810] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:56.654513 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9bd5c-748f-4013-972c-564ca81f81a7 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:56.667655 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9bd5c-748f-4013-972c-564ca81f81a7 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:56.678498 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5edf5-c6e0-4c98-bb74-614e54422f08 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:56.683888 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25ed46-b80d-400e-be57-61cac197e348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:56.684441 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25ed46-b80d-400e-be57-61cac197e348 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:56.684721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25ed46-b80d-400e-be57-61cac197e348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:56.685029 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25ed46-b80d-400e-be57-61cac197e348 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:56.685809 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce25ed46-b80d-400e-be57-61cac197e348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:56.687466 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce25ed46-b80d-400e-be57-61cac197e348 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:56.700728 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 908/1811] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:56.704618 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25ed46-b80d-400e-be57-61cac197e348 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:56.719854 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25ed46-b80d-400e-be57-61cac197e348 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:56.726690 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe8e91-4281-48bc-844d-c1d150ee71b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:56.727491 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe8e91-4281-48bc-844d-c1d150ee71b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:56.727704 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe8e91-4281-48bc-844d-c1d150ee71b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:56.728131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe8e91-4281-48bc-844d-c1d150ee71b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:56.728782 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe8e91-4281-48bc-844d-c1d150ee71b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:56.730650 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe8e91-4281-48bc-844d-c1d150ee71b4 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:56.731870 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f41cd-1fc1-4055-a180-9b9a70de1cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:56.732481 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f41cd-1fc1-4055-a180-9b9a70de1cdd admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:56.733129 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f41cd-1fc1-4055-a180-9b9a70de1cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:56.733410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f41cd-1fc1-4055-a180-9b9a70de1cdd admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:56.734040 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f41cd-1fc1-4055-a180-9b9a70de1cdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:56.736943 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f41cd-1fc1-4055-a180-9b9a70de1cdd admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:56.742739 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 909/1812] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:56.747687 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe8e91-4281-48bc-844d-c1d150ee71b4 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:56.756251 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:56.760256 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe8e91-4281-48bc-844d-c1d150ee71b4 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:56.765917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4cfad-6cae-406e-b94e-d6a18345c9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:56.766647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 904/1813] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:56 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:56.770275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4cfad-6cae-406e-b94e-d6a18345c9b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:56.770635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4cfad-6cae-406e-b94e-d6a18345c9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:56.770960 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4cfad-6cae-406e-b94e-d6a18345c9b2 admin admin] PATH_INFO: `/v3/projects/195488738c3946fdb31545f03eef46a1/users/33ec4c1e4c424466a1654c113993cdb0/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:56.771536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f41cd-1fc1-4055-a180-9b9a70de1cdd admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:56.782061 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f41cd-1fc1-4055-a180-9b9a70de1cdd admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:56.787062 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346e755-29c7-471a-9e3c-8464a894aef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:56.787695 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346e755-29c7-471a-9e3c-8464a894aef2 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:56.787973 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346e755-29c7-471a-9e3c-8464a894aef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:56.788245 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346e755-29c7-471a-9e3c-8464a894aef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:56.788352 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4cfad-6cae-406e-b94e-d6a18345c9b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=195488738c3946fdb31545f03eef46a1, user_id=33ec4c1e4c424466a1654c113993cdb0, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:56.789024 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346e755-29c7-471a-9e3c-8464a894aef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:56.790605 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346e755-29c7-471a-9e3c-8464a894aef2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:56.790982 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4cfad-6cae-406e-b94e-d6a18345c9b2 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:56.804408 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 905/1814] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:56.805585 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346e755-29c7-471a-9e3c-8464a894aef2 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:56.810904 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 910/1815] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:56 2022] PUT /identity/v3/projects/195488738c3946fdb31545f03eef46a1/users/33ec4c1e4c424466a1654c113993cdb0/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:56.816737 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346e755-29c7-471a-9e3c-8464a894aef2 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:56.822674 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c716fe-cc08-411a-8eca-f64f00eceb0b None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:56.822964 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c716fe-cc08-411a-8eca-f64f00eceb0b None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:56.823242 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c716fe-cc08-411a-8eca-f64f00eceb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:56.854305 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-e4c716fe-cc08-411a-8eca-f64f00eceb0b None None] MFA Rules not processed for user `33ec4c1e4c424466a1654c113993cdb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:56.855535 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-e4c716fe-cc08-411a-8eca-f64f00eceb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:56.875946 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:56.900252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952f9b7-8b71-41d0-b926-e5d7db91850d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:56.901036 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952f9b7-8b71-41d0-b926-e5d7db91850d admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:56.901309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952f9b7-8b71-41d0-b926-e5d7db91850d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:56.901629 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952f9b7-8b71-41d0-b926-e5d7db91850d admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:56.902287 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952f9b7-8b71-41d0-b926-e5d7db91850d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:56.903898 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952f9b7-8b71-41d0-b926-e5d7db91850d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:56.915397 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 906/1816] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:56.919574 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952f9b7-8b71-41d0-b926-e5d7db91850d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:56.933572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 911/1817] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:56.945212 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952f9b7-8b71-41d0-b926-e5d7db91850d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:56.953102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c716fe-cc08-411a-8eca-f64f00eceb0b None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:56.955234 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-e4c716fe-cc08-411a-8eca-f64f00eceb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:56.966619 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ee7ff-81d5-43c8-998a-17967f4d7853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:56.967944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ee7ff-81d5-43c8-998a-17967f4d7853 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:56.968893 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ee7ff-81d5-43c8-998a-17967f4d7853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:56.969319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ee7ff-81d5-43c8-998a-17967f4d7853 admin admin] PATH_INFO: `/v3/projects/06cb01135954461d95c7129a7a29926c/users/9eafb82288244cf3b149243de8289a58/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:56.995728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ee7ff-81d5-43c8-998a-17967f4d7853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06cb01135954461d95c7129a7a29926c, user_id=9eafb82288244cf3b149243de8289a58, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:56.998498 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ee7ff-81d5-43c8-998a-17967f4d7853 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:57.021454 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 907/1818] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:56 2022] PUT /identity/v3/projects/06cb01135954461d95c7129a7a29926c/users/9eafb82288244cf3b149243de8289a58/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:36:57.033978 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f53c3-c0f3-481b-b1f9-bc35a27c3e4e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:57.034262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f53c3-c0f3-481b-b1f9-bc35a27c3e4e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:57.034534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f53c3-c0f3-481b-b1f9-bc35a27c3e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:57.045052 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c716fe-cc08-411a-8eca-f64f00eceb0b None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:57.063526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f75f53c3-c0f3-481b-b1f9-bc35a27c3e4e None None] MFA Rules not processed for user `9eafb82288244cf3b149243de8289a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:57.065328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f75f53c3-c0f3-481b-b1f9-bc35a27c3e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:57.096009 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f451039-1b32-40b3-8b73-8afb9c097d33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:57.096623 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f451039-1b32-40b3-8b73-8afb9c097d33 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:57.097073 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f451039-1b32-40b3-8b73-8afb9c097d33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:57.097357 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f451039-1b32-40b3-8b73-8afb9c097d33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:57.132801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 908/1819] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:36:57.145118 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f53c3-c0f3-481b-b1f9-bc35a27c3e4e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:57.154936 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f53c3-c0f3-481b-b1f9-bc35a27c3e4e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:57.158844 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f6832-859e-4946-b037-fb94c1365ac3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:57.159432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f451039-1b32-40b3-8b73-8afb9c097d33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:57.159568 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f6832-859e-4946-b037-fb94c1365ac3 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:57.159761 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f6832-859e-4946-b037-fb94c1365ac3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:57.159982 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f6832-859e-4946-b037-fb94c1365ac3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:57.160907 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f451039-1b32-40b3-8b73-8afb9c097d33 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:57.168393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f6832-859e-4946-b037-fb94c1365ac3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:57.170084 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f6832-859e-4946-b037-fb94c1365ac3 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:57.170196 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 912/1820] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:57.179542 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 909/1821] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:36:57.199753 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f451039-1b32-40b3-8b73-8afb9c097d33 service cinder] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:57.235829 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:57.271232 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f451039-1b32-40b3-8b73-8afb9c097d33 service cinder] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:57.275777 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9dfd35-80c9-405c-bcc0-962025f59f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:57.276408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9dfd35-80c9-405c-bcc0-962025f59f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:57.276688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9dfd35-80c9-405c-bcc0-962025f59f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:57.276996 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9dfd35-80c9-405c-bcc0-962025f59f67 admin admin] PATH_INFO: `/v3/users/9eafb82288244cf3b149243de8289a58` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:57.278124 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9dfd35-80c9-405c-bcc0-962025f59f67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eafb82288244cf3b149243de8289a58)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:57.280054 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9dfd35-80c9-405c-bcc0-962025f59f67 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:57.328259 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ec9dfd35-80c9-405c-bcc0-962025f59f67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eafb82288244cf3b149243de8289a58'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:57.328973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ec9dfd35-80c9-405c-bcc0-962025f59f67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eafb82288244cf3b149243de8289a58'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:57.337431 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ec9dfd35-80c9-405c-bcc0-962025f59f67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eafb82288244cf3b149243de8289a58'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:57.338167 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ec9dfd35-80c9-405c-bcc0-962025f59f67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eafb82288244cf3b149243de8289a58'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:57.348508 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ec9dfd35-80c9-405c-bcc0-962025f59f67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eafb82288244cf3b149243de8289a58'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:57.370059 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 913/1822] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:57 2022] DELETE /identity/v3/users/9eafb82288244cf3b149243de8289a58 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:57.374227 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347ac72-d95b-4f73-8f7f-6102b0d89666 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:57.374875 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347ac72-d95b-4f73-8f7f-6102b0d89666 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:57.374875 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347ac72-d95b-4f73-8f7f-6102b0d89666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:57.398106 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-4347ac72-d95b-4f73-8f7f-6102b0d89666 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:57.399465 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4347ac72-d95b-4f73-8f7f-6102b0d89666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:57.441481 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:57.531468 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 910/1823] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:57.538756 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9dfd35-80c9-405c-bcc0-962025f59f67 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:57.540453 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-ec9dfd35-80c9-405c-bcc0-962025f59f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:57.610826 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9dfd35-80c9-405c-bcc0-962025f59f67 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:57.615529 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf4565-a0c3-45aa-8d68-5c15526f9730 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:57.616193 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf4565-a0c3-45aa-8d68-5c15526f9730 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:57.616477 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf4565-a0c3-45aa-8d68-5c15526f9730 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:57.616754 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf4565-a0c3-45aa-8d68-5c15526f9730 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:57.634244 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf4565-a0c3-45aa-8d68-5c15526f9730 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:57.635895 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cf4565-a0c3-45aa-8d68-5c15526f9730 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:57.647307 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 914/1824] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:57.736711 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347ac72-d95b-4f73-8f7f-6102b0d89666 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:57.738281 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4347ac72-d95b-4f73-8f7f-6102b0d89666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:57.803786 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347ac72-d95b-4f73-8f7f-6102b0d89666 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:57.809348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81fbdbb-9afb-4501-8759-b01520a27cd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:57.809976 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fbdbb-9afb-4501-8759-b01520a27cd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:57.810255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fbdbb-9afb-4501-8759-b01520a27cd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:57.810532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fbdbb-9afb-4501-8759-b01520a27cd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:57.812367 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d81fbdbb-9afb-4501-8759-b01520a27cd7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:57.886235 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81fbdbb-9afb-4501-8759-b01520a27cd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:57.887929 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81fbdbb-9afb-4501-8759-b01520a27cd7 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:57.899252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 911/1825] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:57.953723 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf4565-a0c3-45aa-8d68-5c15526f9730 service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:57.955738 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-70cf4565-a0c3-45aa-8d68-5c15526f9730 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:57.978436 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf4565-a0c3-45aa-8d68-5c15526f9730 service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:57.989004 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf46f30-b3de-4ab2-9a55-c42ec9416418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:57.989723 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf46f30-b3de-4ab2-9a55-c42ec9416418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:57.990315 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf46f30-b3de-4ab2-9a55-c42ec9416418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:57.990645 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf46f30-b3de-4ab2-9a55-c42ec9416418 admin admin] PATH_INFO: `/v3/projects/06cb01135954461d95c7129a7a29926c` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:57.992145 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf46f30-b3de-4ab2-9a55-c42ec9416418 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06cb01135954461d95c7129a7a29926c)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:57.994745 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf46f30-b3de-4ab2-9a55-c42ec9416418 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:58.057757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:58.064323 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5bf46f30-b3de-4ab2-9a55-c42ec9416418 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06cb01135954461d95c7129a7a29926c', 'request_id': 'req-5bf46f30-b3de-4ab2-9a55-c42ec9416418'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:58.082350 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5bf46f30-b3de-4ab2-9a55-c42ec9416418 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06cb01135954461d95c7129a7a29926c', 'request_id': 'req-5bf46f30-b3de-4ab2-9a55-c42ec9416418'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:58.094245 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5bf46f30-b3de-4ab2-9a55-c42ec9416418 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06cb01135954461d95c7129a7a29926c', 'request_id': 'req-5bf46f30-b3de-4ab2-9a55-c42ec9416418'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:58.112453 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5bf46f30-b3de-4ab2-9a55-c42ec9416418 admin admin] The token cache is being invalidate because project 06cb01135954461d95c7129a7a29926c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:36:58.113195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5bf46f30-b3de-4ab2-9a55-c42ec9416418 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:58.113665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5bf46f30-b3de-4ab2-9a55-c42ec9416418 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:58.116052 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 915/1826] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:36:57 2022] DELETE /identity/v3/projects/06cb01135954461d95c7129a7a29926c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:58.120794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0523d-be07-4369-95db-c8631b8c6bcb None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:58.121116 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0523d-be07-4369-95db-c8631b8c6bcb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:58.121398 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0523d-be07-4369-95db-c8631b8c6bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:58.153818 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-42b0523d-be07-4369-95db-c8631b8c6bcb None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:58.155461 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-42b0523d-be07-4369-95db-c8631b8c6bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:58.249853 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 912/1827] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:58.259022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248deeb-32d3-4485-9a2a-6035a8905658 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:58.259470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248deeb-32d3-4485-9a2a-6035a8905658 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:58.259890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6248deeb-32d3-4485-9a2a-6035a8905658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:58.284518 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-6248deeb-32d3-4485-9a2a-6035a8905658 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:58.285845 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-6248deeb-32d3-4485-9a2a-6035a8905658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:58.296200 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 916/1828] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:58.300439 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0523d-be07-4369-95db-c8631b8c6bcb None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:58.311395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0523d-be07-4369-95db-c8631b8c6bcb None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:58.317420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d5bc9-debc-4c50-b9b7-61acf04e834a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:58.318031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d5bc9-debc-4c50-b9b7-61acf04e834a admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:58.318310 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d5bc9-debc-4c50-b9b7-61acf04e834a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:58.318580 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d5bc9-debc-4c50-b9b7-61acf04e834a admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:58.319230 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8d5bc9-debc-4c50-b9b7-61acf04e834a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:36:58.319537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8d5bc9-debc-4c50-b9b7-61acf04e834a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:58.321345 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8d5bc9-debc-4c50-b9b7-61acf04e834a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:58.336900 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 913/1829] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:36:58.339930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248deeb-32d3-4485-9a2a-6035a8905658 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:58.354329 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6248deeb-32d3-4485-9a2a-6035a8905658 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:58.359922 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869fbbb-a64b-4ec9-85d6-8d18569de060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:58.361420 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869fbbb-a64b-4ec9-85d6-8d18569de060 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:58.361420 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869fbbb-a64b-4ec9-85d6-8d18569de060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:58.361645 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869fbbb-a64b-4ec9-85d6-8d18569de060 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:58.362442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869fbbb-a64b-4ec9-85d6-8d18569de060 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:58.364727 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869fbbb-a64b-4ec9-85d6-8d18569de060 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:58.410842 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 917/1830] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:36:58 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:58.419591 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d5bc9-debc-4c50-b9b7-61acf04e834a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:58.432995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d5bc9-debc-4c50-b9b7-61acf04e834a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:58.453797 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:58.558995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f2ccf-0717-47e7-b024-841e395fc1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:58.559795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f2ccf-0717-47e7-b024-841e395fc1ab admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:58.560261 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f2ccf-0717-47e7-b024-841e395fc1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:58.560650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f2ccf-0717-47e7-b024-841e395fc1ab admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:58.561366 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f2ccf-0717-47e7-b024-841e395fc1ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:58.563384 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f2ccf-0717-47e7-b024-841e395fc1ab admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:58.613216 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 914/1831] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:36:58 2022] POST /identity/v3/users => generated 287 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:36:58.622723 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869fbbb-a64b-4ec9-85d6-8d18569de060 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:58.639898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869fbbb-a64b-4ec9-85d6-8d18569de060 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:58.650761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c8052-f6e8-414f-b710-3db6cd8523f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:58.651696 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c8052-f6e8-414f-b710-3db6cd8523f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:58.652182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c8052-f6e8-414f-b710-3db6cd8523f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:58.652691 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c8052-f6e8-414f-b710-3db6cd8523f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:58.653594 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c8052-f6e8-414f-b710-3db6cd8523f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:58.656341 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c8052-f6e8-414f-b710-3db6cd8523f5 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:58.671868 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 918/1832] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:58.677605 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f2ccf-0717-47e7-b024-841e395fc1ab admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:58.695688 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f2ccf-0717-47e7-b024-841e395fc1ab admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:58.702245 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3251b82-3fa8-4584-a964-b4ea96da1780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:58.703558 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3251b82-3fa8-4584-a964-b4ea96da1780 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:58.704051 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3251b82-3fa8-4584-a964-b4ea96da1780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:58.705480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3251b82-3fa8-4584-a964-b4ea96da1780 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:58.707332 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3251b82-3fa8-4584-a964-b4ea96da1780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:58.709602 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3251b82-3fa8-4584-a964-b4ea96da1780 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:58.741058 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 915/1833] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:36:58.745339 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c8052-f6e8-414f-b710-3db6cd8523f5 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:58.759202 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c8052-f6e8-414f-b710-3db6cd8523f5 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:58.777907 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b5ecc-bc6d-4a83-b7ae-bd74a15ec532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:58.778601 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b5ecc-bc6d-4a83-b7ae-bd74a15ec532 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:58.779154 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b5ecc-bc6d-4a83-b7ae-bd74a15ec532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:58.779265 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b5ecc-bc6d-4a83-b7ae-bd74a15ec532 admin admin] PATH_INFO: `/v3/projects/f13ce9c111d74cfe8a8755dec2fb18ac/users/0193afc8d207475c978d6adaab450e9d/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:58.779640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3251b82-3fa8-4584-a964-b4ea96da1780 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:58.783993 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c3251b82-3fa8-4584-a964-b4ea96da1780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:58.796079 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b5ecc-bc6d-4a83-b7ae-bd74a15ec532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f13ce9c111d74cfe8a8755dec2fb18ac, user_id=0193afc8d207475c978d6adaab450e9d, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:58.800261 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b5ecc-bc6d-4a83-b7ae-bd74a15ec532 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:58.827942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 919/1834] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:36:58 2022] PUT /identity/v3/projects/f13ce9c111d74cfe8a8755dec2fb18ac/users/0193afc8d207475c978d6adaab450e9d/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:36:58.839681 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d51517-b99a-4fd1-ba1b-c4c4758cd176 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:58.840962 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d51517-b99a-4fd1-ba1b-c4c4758cd176 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:58.841741 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d51517-b99a-4fd1-ba1b-c4c4758cd176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:58.867060 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3251b82-3fa8-4584-a964-b4ea96da1780 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:58.882639 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-95d51517-b99a-4fd1-ba1b-c4c4758cd176 None None] MFA Rules not processed for user `0193afc8d207475c978d6adaab450e9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:58.884199 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-95d51517-b99a-4fd1-ba1b-c4c4758cd176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:58.946009 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40feb635-abb1-4872-ad45-076948fca79c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:58.946009 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40feb635-abb1-4872-ad45-076948fca79c service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:58.946009 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40feb635-abb1-4872-ad45-076948fca79c service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:58.946009 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40feb635-abb1-4872-ad45-076948fca79c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:58.961114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-40feb635-abb1-4872-ad45-076948fca79c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:58.969247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 920/1835] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:36:59.030564 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d51517-b99a-4fd1-ba1b-c4c4758cd176 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:59.032200 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-95d51517-b99a-4fd1-ba1b-c4c4758cd176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:59.053103 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d51517-b99a-4fd1-ba1b-c4c4758cd176 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:59.057694 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09d661-2da0-4c87-82e0-99a401d8945d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:59.058319 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09d661-2da0-4c87-82e0-99a401d8945d service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:59.058604 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09d661-2da0-4c87-82e0-99a401d8945d service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:59.058886 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09d661-2da0-4c87-82e0-99a401d8945d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:59.067659 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40feb635-abb1-4872-ad45-076948fca79c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:59.069245 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40feb635-abb1-4872-ad45-076948fca79c service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:59.071922 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09d661-2da0-4c87-82e0-99a401d8945d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:59.073792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09d661-2da0-4c87-82e0-99a401d8945d service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:59.082041 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 916/1836] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:36:59.087597 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 921/1837] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:36:59.121613 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40feb635-abb1-4872-ad45-076948fca79c service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:59.126140 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:59.277383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40feb635-abb1-4872-ad45-076948fca79c service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:59.287413 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ea4c6-af54-44a1-9358-4bdb8271409d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:59.288142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ea4c6-af54-44a1-9358-4bdb8271409d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:59.288532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ea4c6-af54-44a1-9358-4bdb8271409d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:59.288889 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ea4c6-af54-44a1-9358-4bdb8271409d admin admin] PATH_INFO: `/v3/users/0193afc8d207475c978d6adaab450e9d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:59.290379 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ea4c6-af54-44a1-9358-4bdb8271409d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0193afc8d207475c978d6adaab450e9d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:59.292539 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ea4c6-af54-44a1-9358-4bdb8271409d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:59.360179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c36ea4c6-af54-44a1-9358-4bdb8271409d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0193afc8d207475c978d6adaab450e9d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:59.360894 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c36ea4c6-af54-44a1-9358-4bdb8271409d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0193afc8d207475c978d6adaab450e9d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:59.370718 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c36ea4c6-af54-44a1-9358-4bdb8271409d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0193afc8d207475c978d6adaab450e9d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:59.382749 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c36ea4c6-af54-44a1-9358-4bdb8271409d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0193afc8d207475c978d6adaab450e9d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:59.386092 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c36ea4c6-af54-44a1-9358-4bdb8271409d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0193afc8d207475c978d6adaab450e9d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:36:59.424827 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 917/1838] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:36:59 2022] DELETE /identity/v3/users/0193afc8d207475c978d6adaab450e9d => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:36:59.429633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3318d-6cd7-48f7-8bce-315080c2d34f None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:59.430189 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3318d-6cd7-48f7-8bce-315080c2d34f None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:59.430270 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3318d-6cd7-48f7-8bce-315080c2d34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:59.459275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-55d3318d-6cd7-48f7-8bce-315080c2d34f None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:36:59.461474 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-55d3318d-6cd7-48f7-8bce-315080c2d34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:59.531876 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:36:59.553493 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 922/1839] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:36:59.563740 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ea4c6-af54-44a1-9358-4bdb8271409d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:36:59.569116 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c36ea4c6-af54-44a1-9358-4bdb8271409d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:36:59.664315 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ea4c6-af54-44a1-9358-4bdb8271409d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:36:59.676710 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba37527-0b1e-4e29-9c05-6f5649914409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:36:59.677403 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba37527-0b1e-4e29-9c05-6f5649914409 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:36:59.677687 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba37527-0b1e-4e29-9c05-6f5649914409 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:36:59.677961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba37527-0b1e-4e29-9c05-6f5649914409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:36:59.695847 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba37527-0b1e-4e29-9c05-6f5649914409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:36:59.700128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba37527-0b1e-4e29-9c05-6f5649914409 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:36:59.735789 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 918/1840] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:36:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:37:00.150620 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3318d-6cd7-48f7-8bce-315080c2d34f None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:00.153275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-55d3318d-6cd7-48f7-8bce-315080c2d34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:00.181540 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3318d-6cd7-48f7-8bce-315080c2d34f None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:00.187350 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589cd15b-00a3-412f-abb7-4bb49efd343c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:00.188108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589cd15b-00a3-412f-abb7-4bb49efd343c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:00.188537 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589cd15b-00a3-412f-abb7-4bb49efd343c admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:00.188959 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589cd15b-00a3-412f-abb7-4bb49efd343c admin admin] PATH_INFO: `/v3/projects/f13ce9c111d74cfe8a8755dec2fb18ac` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:00.196293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589cd15b-00a3-412f-abb7-4bb49efd343c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f13ce9c111d74cfe8a8755dec2fb18ac)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:00.198530 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589cd15b-00a3-412f-abb7-4bb49efd343c admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:00.274481 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-589cd15b-00a3-412f-abb7-4bb49efd343c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f13ce9c111d74cfe8a8755dec2fb18ac', 'request_id': 'req-589cd15b-00a3-412f-abb7-4bb49efd343c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:00.289808 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-589cd15b-00a3-412f-abb7-4bb49efd343c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f13ce9c111d74cfe8a8755dec2fb18ac', 'request_id': 'req-589cd15b-00a3-412f-abb7-4bb49efd343c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:00.307214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-589cd15b-00a3-412f-abb7-4bb49efd343c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f13ce9c111d74cfe8a8755dec2fb18ac', 'request_id': 'req-589cd15b-00a3-412f-abb7-4bb49efd343c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:00.334309 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-589cd15b-00a3-412f-abb7-4bb49efd343c admin admin] The token cache is being invalidate because project f13ce9c111d74cfe8a8755dec2fb18ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:37:00.334860 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-589cd15b-00a3-412f-abb7-4bb49efd343c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:00.335822 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-589cd15b-00a3-412f-abb7-4bb49efd343c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:00.337907 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 923/1841] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:37:00 2022] DELETE /identity/v3/projects/f13ce9c111d74cfe8a8755dec2fb18ac => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:00.343152 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573b584-71d4-4d5f-8a15-009c0415e81d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:00.343463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573b584-71d4-4d5f-8a15-009c0415e81d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:00.343740 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573b584-71d4-4d5f-8a15-009c0415e81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:00.381270 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8573b584-71d4-4d5f-8a15-009c0415e81d None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:00.382796 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8573b584-71d4-4d5f-8a15-009c0415e81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:00.420632 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:00.554222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 919/1842] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:00.563578 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f923d-6429-4acb-9a7a-76ba99adcc58 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:00.564627 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f923d-6429-4acb-9a7a-76ba99adcc58 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:00.564843 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f923d-6429-4acb-9a7a-76ba99adcc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:00.592542 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-780f923d-6429-4acb-9a7a-76ba99adcc58 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:00.594387 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-780f923d-6429-4acb-9a7a-76ba99adcc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:00.620577 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 924/1843] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:00.624820 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8573b584-71d4-4d5f-8a15-009c0415e81d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:00.637690 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8573b584-71d4-4d5f-8a15-009c0415e81d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:00.645023 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce30151-2cf0-4814-9640-b4a9fcae8852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:00.645856 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce30151-2cf0-4814-9640-b4a9fcae8852 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:00.646221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce30151-2cf0-4814-9640-b4a9fcae8852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:00.646567 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce30151-2cf0-4814-9640-b4a9fcae8852 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:00.647375 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce30151-2cf0-4814-9640-b4a9fcae8852 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:37:00.647738 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce30151-2cf0-4814-9640-b4a9fcae8852 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:00.649656 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce30151-2cf0-4814-9640-b4a9fcae8852 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:00.671489 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 920/1844] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:37:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:37:00.671904 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f923d-6429-4acb-9a7a-76ba99adcc58 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:00.686047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f923d-6429-4acb-9a7a-76ba99adcc58 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:00.699554 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce30151-2cf0-4814-9640-b4a9fcae8852 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:00.701110 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-bce30151-2cf0-4814-9640-b4a9fcae8852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:00.701359 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc71d2a-561d-46e2-83b2-be7b9c8bc0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:00.702105 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc71d2a-561d-46e2-83b2-be7b9c8bc0cf admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:00.702479 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc71d2a-561d-46e2-83b2-be7b9c8bc0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:00.702852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc71d2a-561d-46e2-83b2-be7b9c8bc0cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:00.703680 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc71d2a-561d-46e2-83b2-be7b9c8bc0cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:00.706260 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc71d2a-561d-46e2-83b2-be7b9c8bc0cf admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:00.721218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce30151-2cf0-4814-9640-b4a9fcae8852 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:00.729561 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b035c9dc-7743-4923-95b1-0f60df02358c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:00.730209 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035c9dc-7743-4923-95b1-0f60df02358c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:00.730489 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035c9dc-7743-4923-95b1-0f60df02358c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:00.730888 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035c9dc-7743-4923-95b1-0f60df02358c admin admin] PATH_INFO: `/v3/users/33ec4c1e4c424466a1654c113993cdb0` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:00.732164 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b035c9dc-7743-4923-95b1-0f60df02358c admin admin] RBAC: Authorizing `identity:delete_user(user_id=33ec4c1e4c424466a1654c113993cdb0)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:00.734484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b035c9dc-7743-4923-95b1-0f60df02358c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:00.755392 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 925/1845] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:37:00 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:37:00.759867 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc71d2a-561d-46e2-83b2-be7b9c8bc0cf admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:00.801143 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b035c9dc-7743-4923-95b1-0f60df02358c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33ec4c1e4c424466a1654c113993cdb0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:00.802055 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b035c9dc-7743-4923-95b1-0f60df02358c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33ec4c1e4c424466a1654c113993cdb0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:00.813100 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b035c9dc-7743-4923-95b1-0f60df02358c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33ec4c1e4c424466a1654c113993cdb0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:00.831993 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b035c9dc-7743-4923-95b1-0f60df02358c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33ec4c1e4c424466a1654c113993cdb0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:00.832701 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b035c9dc-7743-4923-95b1-0f60df02358c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33ec4c1e4c424466a1654c113993cdb0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:00.854003 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 921/1846] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:37:00 2022] DELETE /identity/v3/users/33ec4c1e4c424466a1654c113993cdb0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:00.858711 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dcebae-427e-4400-9a46-75d0164b0eb6 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:00.860248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dcebae-427e-4400-9a46-75d0164b0eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:00.860531 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dcebae-427e-4400-9a46-75d0164b0eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:00.867506 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc71d2a-561d-46e2-83b2-be7b9c8bc0cf admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:00.886105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c6dcebae-427e-4400-9a46-75d0164b0eb6 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:00.887622 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c6dcebae-427e-4400-9a46-75d0164b0eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:00.922141 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:00.946768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f190f-ab14-44cc-b17e-ed9c90313100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:00.947265 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f190f-ab14-44cc-b17e-ed9c90313100 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:00.947564 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f190f-ab14-44cc-b17e-ed9c90313100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:00.947882 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f190f-ab14-44cc-b17e-ed9c90313100 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:00.948570 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f190f-ab14-44cc-b17e-ed9c90313100 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:00.950643 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f190f-ab14-44cc-b17e-ed9c90313100 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:00.969934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 922/1847] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:37:00.980844 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dcebae-427e-4400-9a46-75d0164b0eb6 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:00.982892 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c6dcebae-427e-4400-9a46-75d0164b0eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:00.995416 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 926/1848] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:37:00 2022] POST /identity/v3/users => generated 293 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:37:00.999627 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f190f-ab14-44cc-b17e-ed9c90313100 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:01.003792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-696f190f-ab14-44cc-b17e-ed9c90313100 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:01.026357 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f190f-ab14-44cc-b17e-ed9c90313100 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:01.027347 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:01.032643 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1513c23-6109-44f8-9a35-1fc80674d41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:01.034162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1513c23-6109-44f8-9a35-1fc80674d41f admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:01.034655 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1513c23-6109-44f8-9a35-1fc80674d41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:01.035007 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1513c23-6109-44f8-9a35-1fc80674d41f admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:01.035840 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1513c23-6109-44f8-9a35-1fc80674d41f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:01.037618 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1513c23-6109-44f8-9a35-1fc80674d41f admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:01.049990 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 927/1849] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:37:01.053718 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1513c23-6109-44f8-9a35-1fc80674d41f admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:01.066289 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1513c23-6109-44f8-9a35-1fc80674d41f admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:01.072570 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dcebae-427e-4400-9a46-75d0164b0eb6 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:01.073067 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d65a2-1770-42ef-9bfa-4aabaff664da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:01.073688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d65a2-1770-42ef-9bfa-4aabaff664da admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:01.074052 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d65a2-1770-42ef-9bfa-4aabaff664da admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:01.074386 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d65a2-1770-42ef-9bfa-4aabaff664da admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:01.075042 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3d65a2-1770-42ef-9bfa-4aabaff664da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:01.076977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c068355-4acb-4064-bbb2-03a68e6be3d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:01.077738 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c068355-4acb-4064-bbb2-03a68e6be3d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:01.078091 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c068355-4acb-4064-bbb2-03a68e6be3d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:01.078424 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c068355-4acb-4064-bbb2-03a68e6be3d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:01.087904 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3d65a2-1770-42ef-9bfa-4aabaff664da admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:01.092009 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c068355-4acb-4064-bbb2-03a68e6be3d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:01.093570 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c068355-4acb-4064-bbb2-03a68e6be3d8 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:01.104694 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 928/1850] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:37:01.108306 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 923/1851] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:37:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:37:01.109440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d65a2-1770-42ef-9bfa-4aabaff664da admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:01.121129 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d65a2-1770-42ef-9bfa-4aabaff664da admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:01.131439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0a0f9-7a5f-4a89-96d6-f2f942a8e5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:01.132135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0a0f9-7a5f-4a89-96d6-f2f942a8e5f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:01.132422 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0a0f9-7a5f-4a89-96d6-f2f942a8e5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:01.132700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0a0f9-7a5f-4a89-96d6-f2f942a8e5f3 admin admin] PATH_INFO: `/v3/projects/5a32b4a204ba4fbd9a3e2cd5551ad591/users/65d857cf186442bdb0f8fd39ea2957dc/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:01.155601 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0a0f9-7a5f-4a89-96d6-f2f942a8e5f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a32b4a204ba4fbd9a3e2cd5551ad591, user_id=65d857cf186442bdb0f8fd39ea2957dc, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:01.160755 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0a0f9-7a5f-4a89-96d6-f2f942a8e5f3 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:01.188995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 929/1852] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:37:01 2022] PUT /identity/v3/projects/5a32b4a204ba4fbd9a3e2cd5551ad591/users/65d857cf186442bdb0f8fd39ea2957dc/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:37:01.193184 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e04a3-57bc-4cab-9d10-ccd32e2cd70d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:01.193593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e04a3-57bc-4cab-9d10-ccd32e2cd70d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:01.194015 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e04a3-57bc-4cab-9d10-ccd32e2cd70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:01.230116 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-740e04a3-57bc-4cab-9d10-ccd32e2cd70d None None] MFA Rules not processed for user `65d857cf186442bdb0f8fd39ea2957dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:01.232918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-740e04a3-57bc-4cab-9d10-ccd32e2cd70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:01.328893 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 924/1853] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:01.340545 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0a0f9-7a5f-4a89-96d6-f2f942a8e5f3 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:01.343090 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-dcb0a0f9-7a5f-4a89-96d6-f2f942a8e5f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:01.443736 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0a0f9-7a5f-4a89-96d6-f2f942a8e5f3 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:01.454283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053228c-a339-4d10-8e52-4712b56ff252 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:01.456389 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053228c-a339-4d10-8e52-4712b56ff252 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:01.456760 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053228c-a339-4d10-8e52-4712b56ff252 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:01.458658 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053228c-a339-4d10-8e52-4712b56ff252 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:01.471696 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053228c-a339-4d10-8e52-4712b56ff252 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:01.476539 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053228c-a339-4d10-8e52-4712b56ff252 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:01.497853 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 930/1854] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:37:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:37:01.745780 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e04a3-57bc-4cab-9d10-ccd32e2cd70d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:01.748295 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-740e04a3-57bc-4cab-9d10-ccd32e2cd70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:01.919010 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e04a3-57bc-4cab-9d10-ccd32e2cd70d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:01.932062 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd8e1e-4314-4ad5-ab9f-553c077bafdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:01.932721 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd8e1e-4314-4ad5-ab9f-553c077bafdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:01.933146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd8e1e-4314-4ad5-ab9f-553c077bafdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:01.933422 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd8e1e-4314-4ad5-ab9f-553c077bafdc admin admin] PATH_INFO: `/v3/projects/195488738c3946fdb31545f03eef46a1` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:01.934582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd8e1e-4314-4ad5-ab9f-553c077bafdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=195488738c3946fdb31545f03eef46a1)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:01.941202 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd8e1e-4314-4ad5-ab9f-553c077bafdc admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:02.076545 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a35384-7230-4f76-a101-2d7356420a7c None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:02.077099 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a35384-7230-4f76-a101-2d7356420a7c None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:02.077225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a35384-7230-4f76-a101-2d7356420a7c None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:02.078668 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 931/1855] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 20 13:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:37:02.080232 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-dedd8e1e-4314-4ad5-ab9f-553c077bafdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '195488738c3946fdb31545f03eef46a1', 'request_id': 'req-dedd8e1e-4314-4ad5-ab9f-553c077bafdc'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:02.089274 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-dedd8e1e-4314-4ad5-ab9f-553c077bafdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '195488738c3946fdb31545f03eef46a1', 'request_id': 'req-dedd8e1e-4314-4ad5-ab9f-553c077bafdc'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:02.093938 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:02.100277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-dedd8e1e-4314-4ad5-ab9f-553c077bafdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '195488738c3946fdb31545f03eef46a1', 'request_id': 'req-dedd8e1e-4314-4ad5-ab9f-553c077bafdc'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:02.114784 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e242d2-ba62-487e-a839-22ac96dc3dd8 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:02.115261 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e242d2-ba62-487e-a839-22ac96dc3dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:02.115437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e242d2-ba62-487e-a839-22ac96dc3dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:02.125242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-dedd8e1e-4314-4ad5-ab9f-553c077bafdc admin admin] The token cache is being invalidate because project 195488738c3946fdb31545f03eef46a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:37:02.125948 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-dedd8e1e-4314-4ad5-ab9f-553c077bafdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:02.126827 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-dedd8e1e-4314-4ad5-ab9f-553c077bafdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:02.129601 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 925/1856] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:37:01 2022] DELETE /identity/v3/projects/195488738c3946fdb31545f03eef46a1 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:02.134293 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b3a63-684d-4e95-b109-79bd9203565d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:02.134955 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b3a63-684d-4e95-b109-79bd9203565d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:02.135453 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b3a63-684d-4e95-b109-79bd9203565d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:02.155932 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-12e242d2-ba62-487e-a839-22ac96dc3dd8 None None] MFA Rules not processed for user `2440594934b64621a01c441133778155`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:02.157470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-12e242d2-ba62-487e-a839-22ac96dc3dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:02.165037 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-6e7b3a63-684d-4e95-b109-79bd9203565d None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:02.166314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6e7b3a63-684d-4e95-b109-79bd9203565d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:02.254036 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 926/1857] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:02.255463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b3a63-684d-4e95-b109-79bd9203565d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:02.256776 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6e7b3a63-684d-4e95-b109-79bd9203565d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:02.268714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 932/1858] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 20 13:37:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:02.269506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b3a63-684d-4e95-b109-79bd9203565d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:02.270608 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d967109-b7f1-4c8a-8b00-5317e4630099 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:02.271070 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d967109-b7f1-4c8a-8b00-5317e4630099 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:02.271532 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d967109-b7f1-4c8a-8b00-5317e4630099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:02.274538 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200424f-c360-4a93-a083-3b751a8adcf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:02.275497 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200424f-c360-4a93-a083-3b751a8adcf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:02.275869 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200424f-c360-4a93-a083-3b751a8adcf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:02.276173 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200424f-c360-4a93-a083-3b751a8adcf1 admin admin] PATH_INFO: `/v3/users/6085945ca3304bc99b4d9ca8a37d5319` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:02.280080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200424f-c360-4a93-a083-3b751a8adcf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6085945ca3304bc99b4d9ca8a37d5319)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:02.280080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200424f-c360-4a93-a083-3b751a8adcf1 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:02.302298 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-5d967109-b7f1-4c8a-8b00-5317e4630099 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:02.303499 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-5d967109-b7f1-4c8a-8b00-5317e4630099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:02.312748 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 933/1859] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:02.316545 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d967109-b7f1-4c8a-8b00-5317e4630099 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:02.318039 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-5d967109-b7f1-4c8a-8b00-5317e4630099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:02.339992 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d967109-b7f1-4c8a-8b00-5317e4630099 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:02.343170 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f200424f-c360-4a93-a083-3b751a8adcf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6085945ca3304bc99b4d9ca8a37d5319'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:02.343961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f200424f-c360-4a93-a083-3b751a8adcf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6085945ca3304bc99b4d9ca8a37d5319'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:02.346726 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e87d8-eeac-4a75-9fbb-1d22e5859379 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2440594934b64621a01c441133778155', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:02.347345 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e87d8-eeac-4a75-9fbb-1d22e5859379 service placement] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:02.347846 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e87d8-eeac-4a75-9fbb-1d22e5859379 service placement] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:02.348138 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e87d8-eeac-4a75-9fbb-1d22e5859379 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:02.350946 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-e52e87d8-eeac-4a75-9fbb-1d22e5859379 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:02.356174 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f200424f-c360-4a93-a083-3b751a8adcf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6085945ca3304bc99b4d9ca8a37d5319'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:02.364935 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f200424f-c360-4a93-a083-3b751a8adcf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6085945ca3304bc99b4d9ca8a37d5319'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:02.365253 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e87d8-eeac-4a75-9fbb-1d22e5859379 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:02.365713 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f200424f-c360-4a93-a083-3b751a8adcf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6085945ca3304bc99b4d9ca8a37d5319'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:02.367331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e87d8-eeac-4a75-9fbb-1d22e5859379 service placement] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:02.380082 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 934/1860] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:37:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:37:02.381120 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e87d8-eeac-4a75-9fbb-1d22e5859379 service placement] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:02.382618 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-e52e87d8-eeac-4a75-9fbb-1d22e5859379 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:02.394588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 927/1861] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:37:02 2022] DELETE /identity/v3/users/6085945ca3304bc99b4d9ca8a37d5319 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:02.402141 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e87d8-eeac-4a75-9fbb-1d22e5859379 service placement] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:02.426423 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:02.500928 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ec4be-31f9-41e3-bc4a-760bd27414d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:02.503175 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ec4be-31f9-41e3-bc4a-760bd27414d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:02.503533 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ec4be-31f9-41e3-bc4a-760bd27414d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:02.503912 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ec4be-31f9-41e3-bc4a-760bd27414d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:02.508921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ec4be-31f9-41e3-bc4a-760bd27414d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:37:02.508921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ec4be-31f9-41e3-bc4a-760bd27414d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:02.508921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ec4be-31f9-41e3-bc4a-760bd27414d8 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:02.523261 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 935/1862] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:37:02.526706 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200424f-c360-4a93-a083-3b751a8adcf1 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:02.545449 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200424f-c360-4a93-a083-3b751a8adcf1 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:02.551387 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6550f3a-a458-4b28-ac15-83f3d4e20568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:02.552550 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6550f3a-a458-4b28-ac15-83f3d4e20568 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:02.552833 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6550f3a-a458-4b28-ac15-83f3d4e20568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:02.553161 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6550f3a-a458-4b28-ac15-83f3d4e20568 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:02.553917 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6550f3a-a458-4b28-ac15-83f3d4e20568 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:02.555838 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6550f3a-a458-4b28-ac15-83f3d4e20568 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:02.595086 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 928/1863] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:37:02 2022] POST /identity/v3/projects => generated 383 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:37:02.595980 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ec4be-31f9-41e3-bc4a-760bd27414d8 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:02.607212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ec4be-31f9-41e3-bc4a-760bd27414d8 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:02.643074 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6550f3a-a458-4b28-ac15-83f3d4e20568 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:02.645151 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b6550f3a-a458-4b28-ac15-83f3d4e20568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:02.687330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ec06b-7f75-4b0c-b23b-95f17a328ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:02.688020 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ec06b-7f75-4b0c-b23b-95f17a328ce1 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:02.689549 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ec06b-7f75-4b0c-b23b-95f17a328ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:02.690384 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ec06b-7f75-4b0c-b23b-95f17a328ce1 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:02.691081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ec06b-7f75-4b0c-b23b-95f17a328ce1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:02.693170 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ec06b-7f75-4b0c-b23b-95f17a328ce1 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:02.741357 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:02.742272 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 936/1864] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:37:02 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:37:02.746317 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ec06b-7f75-4b0c-b23b-95f17a328ce1 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:02.770281 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ec06b-7f75-4b0c-b23b-95f17a328ce1 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:02.778581 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded119d-ef35-4356-b00e-0deb6d570182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:02.779297 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6550f3a-a458-4b28-ac15-83f3d4e20568 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:02.779743 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded119d-ef35-4356-b00e-0deb6d570182 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:02.779743 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded119d-ef35-4356-b00e-0deb6d570182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:02.780080 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded119d-ef35-4356-b00e-0deb6d570182 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:02.780891 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cded119d-ef35-4356-b00e-0deb6d570182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:02.782885 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cded119d-ef35-4356-b00e-0deb6d570182 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:02.786850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb4ff4-28ef-4c62-abf8-2acd3742b04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:02.787847 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb4ff4-28ef-4c62-abf8-2acd3742b04f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:02.788197 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb4ff4-28ef-4c62-abf8-2acd3742b04f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:02.788600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb4ff4-28ef-4c62-abf8-2acd3742b04f admin admin] PATH_INFO: `/v3/projects/4e1ecd40d6914036b80dfcf74eada323` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:02.789972 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb4ff4-28ef-4c62-abf8-2acd3742b04f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e1ecd40d6914036b80dfcf74eada323)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:02.792117 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb4ff4-28ef-4c62-abf8-2acd3742b04f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:02.796779 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 937/1865] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:37:02.800840 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded119d-ef35-4356-b00e-0deb6d570182 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:02.810775 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded119d-ef35-4356-b00e-0deb6d570182 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:02.816887 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c73065-1ccf-423b-8776-60b148cf0343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:02.817534 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c73065-1ccf-423b-8776-60b148cf0343 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:02.817816 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c73065-1ccf-423b-8776-60b148cf0343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:02.818091 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c73065-1ccf-423b-8776-60b148cf0343 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:02.818741 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c73065-1ccf-423b-8776-60b148cf0343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:02.821366 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c73065-1ccf-423b-8776-60b148cf0343 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:02.838110 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 938/1866] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:37:02.839435 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c73065-1ccf-423b-8776-60b148cf0343 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:02.849924 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-dafb4ff4-28ef-4c62-abf8-2acd3742b04f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e1ecd40d6914036b80dfcf74eada323', 'request_id': 'req-dafb4ff4-28ef-4c62-abf8-2acd3742b04f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:02.850536 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c73065-1ccf-423b-8776-60b148cf0343 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:02.856537 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-dafb4ff4-28ef-4c62-abf8-2acd3742b04f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e1ecd40d6914036b80dfcf74eada323', 'request_id': 'req-dafb4ff4-28ef-4c62-abf8-2acd3742b04f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:02.864584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-dafb4ff4-28ef-4c62-abf8-2acd3742b04f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e1ecd40d6914036b80dfcf74eada323', 'request_id': 'req-dafb4ff4-28ef-4c62-abf8-2acd3742b04f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:02.875133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-dafb4ff4-28ef-4c62-abf8-2acd3742b04f admin admin] The token cache is being invalidate because project 4e1ecd40d6914036b80dfcf74eada323 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:37:02.875556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-dafb4ff4-28ef-4c62-abf8-2acd3742b04f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:02.876198 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-dafb4ff4-28ef-4c62-abf8-2acd3742b04f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:02.878728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 929/1867] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:37:02 2022] DELETE /identity/v3/projects/4e1ecd40d6914036b80dfcf74eada323 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:02.883234 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546a170-d632-4581-ad36-98e138747c50 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:02.883522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546a170-d632-4581-ad36-98e138747c50 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:02.883794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546a170-d632-4581-ad36-98e138747c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:02.904702 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f546a170-d632-4581-ad36-98e138747c50 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:02.905992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f546a170-d632-4581-ad36-98e138747c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:02.911790 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68abb3bb-1cb6-4934-8d03-7185e586728c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:02.912468 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abb3bb-1cb6-4934-8d03-7185e586728c admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:02.912749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abb3bb-1cb6-4934-8d03-7185e586728c admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:02.913082 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abb3bb-1cb6-4934-8d03-7185e586728c admin admin] PATH_INFO: `/v3/projects/67afa2b402b64f188c5ab850608c1edf/users/1b22e8a78a0a401fa113d164da37de6c/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:02.929581 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68abb3bb-1cb6-4934-8d03-7185e586728c admin admin] RBAC: Authorizing `identity:create_grant(project_id=67afa2b402b64f188c5ab850608c1edf, user_id=1b22e8a78a0a401fa113d164da37de6c, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:02.932242 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68abb3bb-1cb6-4934-8d03-7185e586728c admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:02.940998 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:02.951177 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 939/1868] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:37:02 2022] PUT /identity/v3/projects/67afa2b402b64f188c5ab850608c1edf/users/1b22e8a78a0a401fa113d164da37de6c/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:37:02.962063 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604dd0ec-6698-46ad-81ac-053bed6970be None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:02.962288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604dd0ec-6698-46ad-81ac-053bed6970be None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:02.962636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604dd0ec-6698-46ad-81ac-053bed6970be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:02.969209 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 930/1869] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:37:02.977475 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf50268-12e8-4a15-9eb3-21753f269fb0 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:02.977765 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf50268-12e8-4a15-9eb3-21753f269fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:02.978040 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf50268-12e8-4a15-9eb3-21753f269fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:02.995259 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-604dd0ec-6698-46ad-81ac-053bed6970be None None] MFA Rules not processed for user `1b22e8a78a0a401fa113d164da37de6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:02.996783 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-604dd0ec-6698-46ad-81ac-053bed6970be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:03.001267 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0bf50268-12e8-4a15-9eb3-21753f269fb0 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:03.003093 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0bf50268-12e8-4a15-9eb3-21753f269fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:03.062523 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 940/1870] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:03.069089 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 931/1871] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:37:03.070004 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604dd0ec-6698-46ad-81ac-053bed6970be None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:03.076190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf50268-12e8-4a15-9eb3-21753f269fb0 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:03.078465 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0bf50268-12e8-4a15-9eb3-21753f269fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:03.081116 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604dd0ec-6698-46ad-81ac-053bed6970be None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:03.086985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8dfbc1-9156-4995-befe-1051eb57372d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:03.087627 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8dfbc1-9156-4995-befe-1051eb57372d admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:03.087911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8dfbc1-9156-4995-befe-1051eb57372d admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:03.088189 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8dfbc1-9156-4995-befe-1051eb57372d admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:03.089186 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8dfbc1-9156-4995-befe-1051eb57372d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:37:03.089539 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8dfbc1-9156-4995-befe-1051eb57372d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:03.089886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:03.092907 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8dfbc1-9156-4995-befe-1051eb57372d admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:03.105909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 941/1872] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:37:03.110153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8dfbc1-9156-4995-befe-1051eb57372d admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:03.122386 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8dfbc1-9156-4995-befe-1051eb57372d admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:03.127993 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b8578-6cb4-47e5-af30-cf3aaeeacfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:03.128602 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b8578-6cb4-47e5-af30-cf3aaeeacfe5 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:03.128921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b8578-6cb4-47e5-af30-cf3aaeeacfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:03.129191 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b8578-6cb4-47e5-af30-cf3aaeeacfe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:03.129822 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b8578-6cb4-47e5-af30-cf3aaeeacfe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:03.131733 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b8578-6cb4-47e5-af30-cf3aaeeacfe5 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:03.149253 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf50268-12e8-4a15-9eb3-21753f269fb0 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:03.154472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84014f8b-bd71-4062-896b-a2a3216e2526 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:03.155589 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84014f8b-bd71-4062-896b-a2a3216e2526 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:03.155825 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84014f8b-bd71-4062-896b-a2a3216e2526 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:03.156176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84014f8b-bd71-4062-896b-a2a3216e2526 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:03.165868 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 942/1873] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:37:03 2022] POST /identity/v3/projects => generated 393 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:37:03.166420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84014f8b-bd71-4062-896b-a2a3216e2526 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:03.167122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b8578-6cb4-47e5-af30-cf3aaeeacfe5 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:03.167908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84014f8b-bd71-4062-896b-a2a3216e2526 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:03.178005 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 932/1874] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:37:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:37:03.243412 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b8578-6cb4-47e5-af30-cf3aaeeacfe5 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:03.249688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eec807-1c9d-4b81-a61a-cfa0576cc06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:03.250303 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eec807-1c9d-4b81-a61a-cfa0576cc06b admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:03.250614 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eec807-1c9d-4b81-a61a-cfa0576cc06b admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:03.250987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eec807-1c9d-4b81-a61a-cfa0576cc06b admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:03.252749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eec807-1c9d-4b81-a61a-cfa0576cc06b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:03.254832 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eec807-1c9d-4b81-a61a-cfa0576cc06b admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:03.278783 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 943/1875] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:37:03 2022] POST /identity/v3/users => generated 297 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:37:03.283014 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84014f8b-bd71-4062-896b-a2a3216e2526 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:03.300786 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84014f8b-bd71-4062-896b-a2a3216e2526 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:03.306302 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55f6b1-b066-4b6d-9914-9a69b382f59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:03.306928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55f6b1-b066-4b6d-9914-9a69b382f59f admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:03.307210 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55f6b1-b066-4b6d-9914-9a69b382f59f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:03.307511 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55f6b1-b066-4b6d-9914-9a69b382f59f admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:03.308129 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b55f6b1-b066-4b6d-9914-9a69b382f59f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:03.310069 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b55f6b1-b066-4b6d-9914-9a69b382f59f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:03.324757 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 933/1876] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:37:03.330171 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eec807-1c9d-4b81-a61a-cfa0576cc06b admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:03.341466 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eec807-1c9d-4b81-a61a-cfa0576cc06b admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:03.349996 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadc3ab-9583-49b2-86aa-568508dca005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:03.350969 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadc3ab-9583-49b2-86aa-568508dca005 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:03.351223 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadc3ab-9583-49b2-86aa-568508dca005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:03.351578 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadc3ab-9583-49b2-86aa-568508dca005 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:03.352233 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eadc3ab-9583-49b2-86aa-568508dca005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:03.354058 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eadc3ab-9583-49b2-86aa-568508dca005 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:03.367296 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 944/1877] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:37:03.372283 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55f6b1-b066-4b6d-9914-9a69b382f59f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:03.384957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55f6b1-b066-4b6d-9914-9a69b382f59f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:03.390184 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f643f2d-2a66-4406-bfc3-5682f9c27ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:03.390869 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f643f2d-2a66-4406-bfc3-5682f9c27ba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:03.391146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f643f2d-2a66-4406-bfc3-5682f9c27ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:03.391418 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f643f2d-2a66-4406-bfc3-5682f9c27ba3 admin admin] PATH_INFO: `/v3/projects/f36e9fb394944eb0a0c2c0075dbc937c/users/2dd45d3b097c46c28eb94c5e0bd2a7a6/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:03.410098 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f643f2d-2a66-4406-bfc3-5682f9c27ba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f36e9fb394944eb0a0c2c0075dbc937c, user_id=2dd45d3b097c46c28eb94c5e0bd2a7a6, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:03.412989 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f643f2d-2a66-4406-bfc3-5682f9c27ba3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:03.431143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadc3ab-9583-49b2-86aa-568508dca005 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:03.433138 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 934/1878] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:37:03 2022] PUT /identity/v3/projects/f36e9fb394944eb0a0c2c0075dbc937c/users/2dd45d3b097c46c28eb94c5e0bd2a7a6/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:03.433263 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-8eadc3ab-9583-49b2-86aa-568508dca005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:03.438283 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d57b5-a82c-4459-8308-94ce5fe963ca None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:03.438650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d57b5-a82c-4459-8308-94ce5fe963ca None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:03.439019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d57b5-a82c-4459-8308-94ce5fe963ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:03.447718 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eadc3ab-9583-49b2-86aa-568508dca005 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:03.472788 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a29d57b5-a82c-4459-8308-94ce5fe963ca None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:03.474176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a29d57b5-a82c-4459-8308-94ce5fe963ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:03.480034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:03.530555 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bf733-e0f7-4610-b09b-5245a90ef31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:03.531161 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bf733-e0f7-4610-b09b-5245a90ef31b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:03.531473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bf733-e0f7-4610-b09b-5245a90ef31b admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:03.531746 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bf733-e0f7-4610-b09b-5245a90ef31b admin admin] PATH_INFO: `/v3/users/3d633b9b875f47fb841091cee5f0a1db` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:03.534448 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660bf733-e0f7-4610-b09b-5245a90ef31b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d633b9b875f47fb841091cee5f0a1db)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:03.537052 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660bf733-e0f7-4610-b09b-5245a90ef31b admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:03.579420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 935/1879] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:03.589101 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d57b5-a82c-4459-8308-94ce5fe963ca None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:03.590904 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a29d57b5-a82c-4459-8308-94ce5fe963ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:03.596275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-660bf733-e0f7-4610-b09b-5245a90ef31b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d633b9b875f47fb841091cee5f0a1db'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:03.597301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-660bf733-e0f7-4610-b09b-5245a90ef31b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d633b9b875f47fb841091cee5f0a1db'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:03.605115 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-660bf733-e0f7-4610-b09b-5245a90ef31b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d633b9b875f47fb841091cee5f0a1db'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:03.605786 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-660bf733-e0f7-4610-b09b-5245a90ef31b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d633b9b875f47fb841091cee5f0a1db'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:03.613876 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-660bf733-e0f7-4610-b09b-5245a90ef31b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d633b9b875f47fb841091cee5f0a1db'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:03.633243 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 945/1880] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:37:03 2022] DELETE /identity/v3/users/3d633b9b875f47fb841091cee5f0a1db => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:03.637507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82843dcd-c562-4b24-980f-0d87d027dc2b None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:03.637795 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82843dcd-c562-4b24-980f-0d87d027dc2b None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:03.638071 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82843dcd-c562-4b24-980f-0d87d027dc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:03.652853 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d57b5-a82c-4459-8308-94ce5fe963ca None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:03.658323 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-82843dcd-c562-4b24-980f-0d87d027dc2b None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:03.659517 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-82843dcd-c562-4b24-980f-0d87d027dc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:03.682885 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:03.705313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff4fbd5-d415-4e03-957b-b59104b676b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:03.705930 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4fbd5-d415-4e03-957b-b59104b676b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:03.706208 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4fbd5-d415-4e03-957b-b59104b676b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:03.706476 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff4fbd5-d415-4e03-957b-b59104b676b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:03.708276 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6ff4fbd5-d415-4e03-957b-b59104b676b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:03.720720 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 946/1881] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:37:03.730170 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82843dcd-c562-4b24-980f-0d87d027dc2b None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:03.731595 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-82843dcd-c562-4b24-980f-0d87d027dc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:03.743808 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82843dcd-c562-4b24-980f-0d87d027dc2b None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:03.747536 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173360b-6aa8-4474-b141-75a7640667d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:03.748148 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173360b-6aa8-4474-b141-75a7640667d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:03.748457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173360b-6aa8-4474-b141-75a7640667d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:03.748700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173360b-6aa8-4474-b141-75a7640667d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:03.759987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0173360b-6aa8-4474-b141-75a7640667d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:03.761626 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0173360b-6aa8-4474-b141-75a7640667d6 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:03.772912 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 947/1882] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:37:03.782454 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff4fbd5-d415-4e03-957b-b59104b676b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:03.784140 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff4fbd5-d415-4e03-957b-b59104b676b7 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:03.794964 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 936/1883] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:37:04.167614 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173360b-6aa8-4474-b141-75a7640667d6 service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:04.169261 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-0173360b-6aa8-4474-b141-75a7640667d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:04.188662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173360b-6aa8-4474-b141-75a7640667d6 service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:04.194261 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e352aa9-0cae-4a98-9a47-980a711c5980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:04.195222 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e352aa9-0cae-4a98-9a47-980a711c5980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:04.195586 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e352aa9-0cae-4a98-9a47-980a711c5980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:04.195937 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e352aa9-0cae-4a98-9a47-980a711c5980 admin admin] PATH_INFO: `/v3/projects/3e50454e1f9946d9aedcd21f8ebd58b3` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:04.197209 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e352aa9-0cae-4a98-9a47-980a711c5980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e50454e1f9946d9aedcd21f8ebd58b3)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:04.199228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e352aa9-0cae-4a98-9a47-980a711c5980 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:04.239920 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:04.260223 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-1e352aa9-0cae-4a98-9a47-980a711c5980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e50454e1f9946d9aedcd21f8ebd58b3', 'request_id': 'req-1e352aa9-0cae-4a98-9a47-980a711c5980'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:04.268339 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-1e352aa9-0cae-4a98-9a47-980a711c5980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e50454e1f9946d9aedcd21f8ebd58b3', 'request_id': 'req-1e352aa9-0cae-4a98-9a47-980a711c5980'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:04.278623 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-1e352aa9-0cae-4a98-9a47-980a711c5980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e50454e1f9946d9aedcd21f8ebd58b3', 'request_id': 'req-1e352aa9-0cae-4a98-9a47-980a711c5980'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:04.291345 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-1e352aa9-0cae-4a98-9a47-980a711c5980 admin admin] The token cache is being invalidate because project 3e50454e1f9946d9aedcd21f8ebd58b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:37:04.291638 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-1e352aa9-0cae-4a98-9a47-980a711c5980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:04.292296 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-1e352aa9-0cae-4a98-9a47-980a711c5980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:04.293827 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 948/1884] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:37:04 2022] DELETE /identity/v3/projects/3e50454e1f9946d9aedcd21f8ebd58b3 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:04.297948 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff4fbd5-d415-4e03-957b-b59104b676b7 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:04.309022 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff4fbd5-d415-4e03-957b-b59104b676b7 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:04.373927 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc2053-a776-467b-a847-6475057b8dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:04.374589 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc2053-a776-467b-a847-6475057b8dcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:04.374862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc2053-a776-467b-a847-6475057b8dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:04.375137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc2053-a776-467b-a847-6475057b8dcf admin admin] PATH_INFO: `/v3/users/68a7a9ac0445481186518adb11cdbf6c` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:04.376821 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc2053-a776-467b-a847-6475057b8dcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=68a7a9ac0445481186518adb11cdbf6c)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:04.378981 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc2053-a776-467b-a847-6475057b8dcf admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:04.427486 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c5fc2053-a776-467b-a847-6475057b8dcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68a7a9ac0445481186518adb11cdbf6c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:04.428141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c5fc2053-a776-467b-a847-6475057b8dcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68a7a9ac0445481186518adb11cdbf6c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:04.435519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c5fc2053-a776-467b-a847-6475057b8dcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68a7a9ac0445481186518adb11cdbf6c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:04.445319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c5fc2053-a776-467b-a847-6475057b8dcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68a7a9ac0445481186518adb11cdbf6c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:04.446182 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c5fc2053-a776-467b-a847-6475057b8dcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68a7a9ac0445481186518adb11cdbf6c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:04.465934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 937/1885] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:37:04 2022] DELETE /identity/v3/users/68a7a9ac0445481186518adb11cdbf6c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:37:04.782257 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e352aa9-0cae-4a98-9a47-980a711c5980 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:04.785036 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-1e352aa9-0cae-4a98-9a47-980a711c5980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:04.810197 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e352aa9-0cae-4a98-9a47-980a711c5980 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:04.896127 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51b3c8-7cd6-43c9-b45d-6c2b5fe6d7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:04.896402 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51b3c8-7cd6-43c9-b45d-6c2b5fe6d7b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:04.896814 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51b3c8-7cd6-43c9-b45d-6c2b5fe6d7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:04.897200 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51b3c8-7cd6-43c9-b45d-6c2b5fe6d7b5 admin admin] PATH_INFO: `/v3/projects/f0ef9819949e4b1d938213eee8db72de` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:04.898498 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51b3c8-7cd6-43c9-b45d-6c2b5fe6d7b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0ef9819949e4b1d938213eee8db72de)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:04.900534 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51b3c8-7cd6-43c9-b45d-6c2b5fe6d7b5 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:04.966483 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-9c51b3c8-7cd6-43c9-b45d-6c2b5fe6d7b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0ef9819949e4b1d938213eee8db72de', 'request_id': 'req-9c51b3c8-7cd6-43c9-b45d-6c2b5fe6d7b5'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:04.974398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-9c51b3c8-7cd6-43c9-b45d-6c2b5fe6d7b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0ef9819949e4b1d938213eee8db72de', 'request_id': 'req-9c51b3c8-7cd6-43c9-b45d-6c2b5fe6d7b5'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:04.984177 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-9c51b3c8-7cd6-43c9-b45d-6c2b5fe6d7b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0ef9819949e4b1d938213eee8db72de', 'request_id': 'req-9c51b3c8-7cd6-43c9-b45d-6c2b5fe6d7b5'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:04.996106 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-9c51b3c8-7cd6-43c9-b45d-6c2b5fe6d7b5 admin admin] The token cache is being invalidate because project f0ef9819949e4b1d938213eee8db72de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:37:04.996446 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-9c51b3c8-7cd6-43c9-b45d-6c2b5fe6d7b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:04.997512 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-9c51b3c8-7cd6-43c9-b45d-6c2b5fe6d7b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:04.999326 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 949/1886] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:37:04 2022] DELETE /identity/v3/projects/f0ef9819949e4b1d938213eee8db72de => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:05.004370 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc2053-a776-467b-a847-6475057b8dcf admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:05.005872 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c5fc2053-a776-467b-a847-6475057b8dcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:05.013983 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:05.019280 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc2053-a776-467b-a847-6475057b8dcf admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:05.099096 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc530339-5b3e-41a5-bdde-328307b86081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:05.099822 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc530339-5b3e-41a5-bdde-328307b86081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:05.100735 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc530339-5b3e-41a5-bdde-328307b86081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:05.102942 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc530339-5b3e-41a5-bdde-328307b86081 admin admin] PATH_INFO: `/v3/users/e0aee576ef3444e79bead251b40c1df9` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:05.105182 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc530339-5b3e-41a5-bdde-328307b86081 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0aee576ef3444e79bead251b40c1df9)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:05.107555 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc530339-5b3e-41a5-bdde-328307b86081 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:05.186203 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fc530339-5b3e-41a5-bdde-328307b86081 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0aee576ef3444e79bead251b40c1df9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:05.186906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fc530339-5b3e-41a5-bdde-328307b86081 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0aee576ef3444e79bead251b40c1df9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:05.194258 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fc530339-5b3e-41a5-bdde-328307b86081 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0aee576ef3444e79bead251b40c1df9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:05.203960 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fc530339-5b3e-41a5-bdde-328307b86081 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0aee576ef3444e79bead251b40c1df9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:05.205176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fc530339-5b3e-41a5-bdde-328307b86081 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0aee576ef3444e79bead251b40c1df9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:05.227970 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 938/1887] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:37:05 2022] DELETE /identity/v3/users/e0aee576ef3444e79bead251b40c1df9 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:05.580745 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51b3c8-7cd6-43c9-b45d-6c2b5fe6d7b5 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:05.582750 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-9c51b3c8-7cd6-43c9-b45d-6c2b5fe6d7b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:05.598222 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51b3c8-7cd6-43c9-b45d-6c2b5fe6d7b5 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:05.668469 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:05.680717 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896fd9e-bfa9-4e11-8f62-4d70e51ca00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:05.682354 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896fd9e-bfa9-4e11-8f62-4d70e51ca00a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:05.682725 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896fd9e-bfa9-4e11-8f62-4d70e51ca00a admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:05.683069 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896fd9e-bfa9-4e11-8f62-4d70e51ca00a admin admin] PATH_INFO: `/v3/projects/974af7d6480947cea2ced410115d5f1b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:05.684422 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896fd9e-bfa9-4e11-8f62-4d70e51ca00a admin admin] RBAC: Authorizing `identity:delete_project(project_id=974af7d6480947cea2ced410115d5f1b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:05.686503 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896fd9e-bfa9-4e11-8f62-4d70e51ca00a admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:05.764116 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0896fd9e-bfa9-4e11-8f62-4d70e51ca00a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '974af7d6480947cea2ced410115d5f1b', 'request_id': 'req-0896fd9e-bfa9-4e11-8f62-4d70e51ca00a'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:05.771386 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0896fd9e-bfa9-4e11-8f62-4d70e51ca00a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '974af7d6480947cea2ced410115d5f1b', 'request_id': 'req-0896fd9e-bfa9-4e11-8f62-4d70e51ca00a'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:05.779188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0896fd9e-bfa9-4e11-8f62-4d70e51ca00a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '974af7d6480947cea2ced410115d5f1b', 'request_id': 'req-0896fd9e-bfa9-4e11-8f62-4d70e51ca00a'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:05.792298 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0896fd9e-bfa9-4e11-8f62-4d70e51ca00a admin admin] The token cache is being invalidate because project 974af7d6480947cea2ced410115d5f1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:37:05.792595 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0896fd9e-bfa9-4e11-8f62-4d70e51ca00a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:05.793282 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0896fd9e-bfa9-4e11-8f62-4d70e51ca00a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:05.794847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 950/1888] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:37:05 2022] DELETE /identity/v3/projects/974af7d6480947cea2ced410115d5f1b => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:05.799681 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551467b1-6c96-47df-a067-f7a0ccc659f2 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:05.799970 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551467b1-6c96-47df-a067-f7a0ccc659f2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:05.800251 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551467b1-6c96-47df-a067-f7a0ccc659f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:05.820500 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-551467b1-6c96-47df-a067-f7a0ccc659f2 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:05.821741 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-551467b1-6c96-47df-a067-f7a0ccc659f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:05.885338 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 939/1889] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:05.894849 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75191b5e-a60c-4467-88d2-7dcf74e6293b None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:05.895137 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75191b5e-a60c-4467-88d2-7dcf74e6293b None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:05.895416 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75191b5e-a60c-4467-88d2-7dcf74e6293b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:05.917291 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-75191b5e-a60c-4467-88d2-7dcf74e6293b None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:05.918539 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-75191b5e-a60c-4467-88d2-7dcf74e6293b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:05.927766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 951/1890] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:05.931986 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551467b1-6c96-47df-a067-f7a0ccc659f2 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:05.942182 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551467b1-6c96-47df-a067-f7a0ccc659f2 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:05.947166 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe0f70-3cb3-45a2-b0e0-5b4dea09e651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:05.947768 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe0f70-3cb3-45a2-b0e0-5b4dea09e651 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:05.948075 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe0f70-3cb3-45a2-b0e0-5b4dea09e651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:05.948345 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe0f70-3cb3-45a2-b0e0-5b4dea09e651 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:05.949039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe0f70-3cb3-45a2-b0e0-5b4dea09e651 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:37:05.949352 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe0f70-3cb3-45a2-b0e0-5b4dea09e651 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:05.950959 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe0f70-3cb3-45a2-b0e0-5b4dea09e651 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:05.962668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 940/1891] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:37:05.967165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75191b5e-a60c-4467-88d2-7dcf74e6293b None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:05.977384 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75191b5e-a60c-4467-88d2-7dcf74e6293b None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:05.982302 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd866f3c-b5ea-44cb-9f1a-f5673ba1af00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:05.982893 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd866f3c-b5ea-44cb-9f1a-f5673ba1af00 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:05.983176 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd866f3c-b5ea-44cb-9f1a-f5673ba1af00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:05.983453 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd866f3c-b5ea-44cb-9f1a-f5673ba1af00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:05.984237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd866f3c-b5ea-44cb-9f1a-f5673ba1af00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:05.986548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd866f3c-b5ea-44cb-9f1a-f5673ba1af00 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:06.016610 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 952/1892] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:37:05 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:37:06.021261 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe0f70-3cb3-45a2-b0e0-5b4dea09e651 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:06.031084 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe0f70-3cb3-45a2-b0e0-5b4dea09e651 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:06.037411 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:06.102499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc7eab3-b4ea-4274-87f9-c35ee772e9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:06.103119 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7eab3-b4ea-4274-87f9-c35ee772e9c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:06.103400 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7eab3-b4ea-4274-87f9-c35ee772e9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:06.103674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7eab3-b4ea-4274-87f9-c35ee772e9c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:06.104309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc7eab3-b4ea-4274-87f9-c35ee772e9c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:06.106285 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc7eab3-b4ea-4274-87f9-c35ee772e9c9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:06.136433 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 941/1893] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:37:06 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:37:06.137614 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd866f3c-b5ea-44cb-9f1a-f5673ba1af00 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:06.153332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd866f3c-b5ea-44cb-9f1a-f5673ba1af00 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:06.161585 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844b29e-eca1-4279-b2cc-26faf045b2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:06.162266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844b29e-eca1-4279-b2cc-26faf045b2a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:06.162459 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844b29e-eca1-4279-b2cc-26faf045b2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:06.162739 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844b29e-eca1-4279-b2cc-26faf045b2a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:06.163407 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b844b29e-eca1-4279-b2cc-26faf045b2a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:06.165097 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b844b29e-eca1-4279-b2cc-26faf045b2a1 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:06.179275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 953/1894] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:37:06.180546 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc7eab3-b4ea-4274-87f9-c35ee772e9c9 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:06.212298 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc7eab3-b4ea-4274-87f9-c35ee772e9c9 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:06.217893 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73e7ed-9966-4c12-8c17-dcbe2ae1e921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:06.218527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73e7ed-9966-4c12-8c17-dcbe2ae1e921 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:06.218798 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73e7ed-9966-4c12-8c17-dcbe2ae1e921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:06.219067 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73e7ed-9966-4c12-8c17-dcbe2ae1e921 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:06.219717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73e7ed-9966-4c12-8c17-dcbe2ae1e921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:06.222870 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73e7ed-9966-4c12-8c17-dcbe2ae1e921 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:06.238068 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 942/1895] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:37:06.239257 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844b29e-eca1-4279-b2cc-26faf045b2a1 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:06.253171 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844b29e-eca1-4279-b2cc-26faf045b2a1 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:06.263957 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05381f97-07f3-4061-a5e8-5c547a3ea21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:06.265604 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05381f97-07f3-4061-a5e8-5c547a3ea21d admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:06.266080 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05381f97-07f3-4061-a5e8-5c547a3ea21d admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:06.266342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05381f97-07f3-4061-a5e8-5c547a3ea21d admin admin] PATH_INFO: `/v3/projects/e1f267f9502b437495d396d31e68c0c9/users/7b83f9b50c6d44e7b2e0ef7d8480177c/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:06.286153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05381f97-07f3-4061-a5e8-5c547a3ea21d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1f267f9502b437495d396d31e68c0c9, user_id=7b83f9b50c6d44e7b2e0ef7d8480177c, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:06.289147 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05381f97-07f3-4061-a5e8-5c547a3ea21d admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:06.304380 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 954/1896] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:37:06 2022] PUT /identity/v3/projects/e1f267f9502b437495d396d31e68c0c9/users/7b83f9b50c6d44e7b2e0ef7d8480177c/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:06.315095 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9fa62-8915-4c5c-bd50-bd0a7b10ce2e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:06.315523 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9fa62-8915-4c5c-bd50-bd0a7b10ce2e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:06.315877 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9fa62-8915-4c5c-bd50-bd0a7b10ce2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:06.351949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-70a9fa62-8915-4c5c-bd50-bd0a7b10ce2e None None] MFA Rules not processed for user `7b83f9b50c6d44e7b2e0ef7d8480177c`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:06.353309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-70a9fa62-8915-4c5c-bd50-bd0a7b10ce2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:06.437732 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 943/1897] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:06.449987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05381f97-07f3-4061-a5e8-5c547a3ea21d admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:06.451890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-05381f97-07f3-4061-a5e8-5c547a3ea21d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:06.530375 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05381f97-07f3-4061-a5e8-5c547a3ea21d admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:06.536831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1eb940-81c5-4243-a2db-a71a08719908 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:06.537595 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1eb940-81c5-4243-a2db-a71a08719908 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:06.537982 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1eb940-81c5-4243-a2db-a71a08719908 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:06.538414 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1eb940-81c5-4243-a2db-a71a08719908 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:06.548550 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1eb940-81c5-4243-a2db-a71a08719908 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:06.550039 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1eb940-81c5-4243-a2db-a71a08719908 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:06.560020 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 955/1898] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:37:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:37:06.586075 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9fa62-8915-4c5c-bd50-bd0a7b10ce2e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:06.663398 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9fa62-8915-4c5c-bd50-bd0a7b10ce2e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:06.668967 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c41780c-aa4b-48e7-a209-0fb0e4b56ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:06.669693 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41780c-aa4b-48e7-a209-0fb0e4b56ec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:06.669989 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41780c-aa4b-48e7-a209-0fb0e4b56ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:06.670250 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41780c-aa4b-48e7-a209-0fb0e4b56ec2 admin admin] PATH_INFO: `/v3/users/7b83f9b50c6d44e7b2e0ef7d8480177c` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:06.672518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c41780c-aa4b-48e7-a209-0fb0e4b56ec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b83f9b50c6d44e7b2e0ef7d8480177c)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:06.675166 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c41780c-aa4b-48e7-a209-0fb0e4b56ec2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:06.701749 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:06.766338 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5c41780c-aa4b-48e7-a209-0fb0e4b56ec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b83f9b50c6d44e7b2e0ef7d8480177c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:06.769573 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5c41780c-aa4b-48e7-a209-0fb0e4b56ec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b83f9b50c6d44e7b2e0ef7d8480177c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:06.790101 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5c41780c-aa4b-48e7-a209-0fb0e4b56ec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b83f9b50c6d44e7b2e0ef7d8480177c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:06.811228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5c41780c-aa4b-48e7-a209-0fb0e4b56ec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b83f9b50c6d44e7b2e0ef7d8480177c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:06.812233 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5c41780c-aa4b-48e7-a209-0fb0e4b56ec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b83f9b50c6d44e7b2e0ef7d8480177c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:06.851514 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 944/1899] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:37:06 2022] DELETE /identity/v3/users/7b83f9b50c6d44e7b2e0ef7d8480177c => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:06.855871 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9df9c-557c-41e9-9106-debe648b951c None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:06.856329 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9df9c-557c-41e9-9106-debe648b951c None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:06.856431 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9df9c-557c-41e9-9106-debe648b951c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:06.892969 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-10b9df9c-557c-41e9-9106-debe648b951c None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:06.894204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-10b9df9c-557c-41e9-9106-debe648b951c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:06.930594 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:07.034176 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 956/1900] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:07.042253 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c41780c-aa4b-48e7-a209-0fb0e4b56ec2 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:07.043846 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5c41780c-aa4b-48e7-a209-0fb0e4b56ec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:07.115123 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c41780c-aa4b-48e7-a209-0fb0e4b56ec2 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:07.119458 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af84e9-98b2-4e3f-90ff-5e5b7f02cdfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:07.119918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af84e9-98b2-4e3f-90ff-5e5b7f02cdfa service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:07.120444 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af84e9-98b2-4e3f-90ff-5e5b7f02cdfa service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:07.120817 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af84e9-98b2-4e3f-90ff-5e5b7f02cdfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:07.131799 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af84e9-98b2-4e3f-90ff-5e5b7f02cdfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:07.134176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af84e9-98b2-4e3f-90ff-5e5b7f02cdfa service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:07.149457 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 945/1901] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:37:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:37:07.529607 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9df9c-557c-41e9-9106-debe648b951c None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:07.531091 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-10b9df9c-557c-41e9-9106-debe648b951c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:07.551677 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9df9c-557c-41e9-9106-debe648b951c None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:07.563019 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b0f44-4fac-46ce-953f-ee00c573013d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:07.563774 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b0f44-4fac-46ce-953f-ee00c573013d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:07.564067 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b0f44-4fac-46ce-953f-ee00c573013d admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:07.564498 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b0f44-4fac-46ce-953f-ee00c573013d admin admin] PATH_INFO: `/v3/projects/e1f267f9502b437495d396d31e68c0c9` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:07.567415 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b0f44-4fac-46ce-953f-ee00c573013d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1f267f9502b437495d396d31e68c0c9)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:07.572716 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b0f44-4fac-46ce-953f-ee00c573013d admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:07.671735 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-074b0f44-4fac-46ce-953f-ee00c573013d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1f267f9502b437495d396d31e68c0c9', 'request_id': 'req-074b0f44-4fac-46ce-953f-ee00c573013d'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:07.682181 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-074b0f44-4fac-46ce-953f-ee00c573013d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1f267f9502b437495d396d31e68c0c9', 'request_id': 'req-074b0f44-4fac-46ce-953f-ee00c573013d'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:07.693906 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-074b0f44-4fac-46ce-953f-ee00c573013d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1f267f9502b437495d396d31e68c0c9', 'request_id': 'req-074b0f44-4fac-46ce-953f-ee00c573013d'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:07.714578 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-074b0f44-4fac-46ce-953f-ee00c573013d admin admin] The token cache is being invalidate because project e1f267f9502b437495d396d31e68c0c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:37:07.715094 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-074b0f44-4fac-46ce-953f-ee00c573013d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:07.715923 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-074b0f44-4fac-46ce-953f-ee00c573013d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:07.729154 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 957/1902] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:37:07 2022] DELETE /identity/v3/projects/e1f267f9502b437495d396d31e68c0c9 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:08.725398 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f8f8e-9ace-4d6c-9f49-196872a495cc None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:08.726216 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f8f8e-9ace-4d6c-9f49-196872a495cc None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:08.726492 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f8f8e-9ace-4d6c-9f49-196872a495cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:08.757890 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-150f8f8e-9ace-4d6c-9f49-196872a495cc None None] MFA Rules not processed for user `2dd45d3b097c46c28eb94c5e0bd2a7a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:08.759157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-150f8f8e-9ace-4d6c-9f49-196872a495cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:08.863422 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 946/1903] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:08.869676 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad8d25-7d0e-432a-b3b2-191a34d9832e None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:08.870198 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad8d25-7d0e-432a-b3b2-191a34d9832e None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:08.870285 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ad8d25-7d0e-432a-b3b2-191a34d9832e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:08.890780 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-27ad8d25-7d0e-432a-b3b2-191a34d9832e None None] MFA Rules not processed for user `2dd45d3b097c46c28eb94c5e0bd2a7a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:08.892111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-27ad8d25-7d0e-432a-b3b2-191a34d9832e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:08.899437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 958/1904] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:08.913365 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f8f8e-9ace-4d6c-9f49-196872a495cc None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:08.915004 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-150f8f8e-9ace-4d6c-9f49-196872a495cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:08.951719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:08.998484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f8f8e-9ace-4d6c-9f49-196872a495cc None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:09.005033 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe4b74-df65-481a-85f0-51184b8af13f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:09.005670 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe4b74-df65-481a-85f0-51184b8af13f service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:09.005955 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe4b74-df65-481a-85f0-51184b8af13f service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:09.006231 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe4b74-df65-481a-85f0-51184b8af13f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:09.014977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe4b74-df65-481a-85f0-51184b8af13f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:09.016608 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe4b74-df65-481a-85f0-51184b8af13f service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:09.026769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 947/1905] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:37:08 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:37:09.753290 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f6f9e-873e-42a9-a31e-a2ef7ca280ca None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:09.753622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f6f9e-873e-42a9-a31e-a2ef7ca280ca None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:09.754038 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f6f9e-873e-42a9-a31e-a2ef7ca280ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:09.779297 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-284f6f9e-873e-42a9-a31e-a2ef7ca280ca None None] MFA Rules not processed for user `2dd45d3b097c46c28eb94c5e0bd2a7a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:09.780945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-284f6f9e-873e-42a9-a31e-a2ef7ca280ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:09.790034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 959/1906] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:37:10.215787 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe4b74-df65-481a-85f0-51184b8af13f service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:10.217411 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-63fe4b74-df65-481a-85f0-51184b8af13f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:10.310469 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe4b74-df65-481a-85f0-51184b8af13f service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:10.316245 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89492bb-103d-4bb2-a5b1-8e8846065bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:10.316920 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89492bb-103d-4bb2-a5b1-8e8846065bb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:10.317207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89492bb-103d-4bb2-a5b1-8e8846065bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:10.317601 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89492bb-103d-4bb2-a5b1-8e8846065bb3 admin admin] PATH_INFO: `/v3/users/1b22e8a78a0a401fa113d164da37de6c` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:10.318608 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89492bb-103d-4bb2-a5b1-8e8846065bb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b22e8a78a0a401fa113d164da37de6c)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:10.320559 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89492bb-103d-4bb2-a5b1-8e8846065bb3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:10.388269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c89492bb-103d-4bb2-a5b1-8e8846065bb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b22e8a78a0a401fa113d164da37de6c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:10.389317 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c89492bb-103d-4bb2-a5b1-8e8846065bb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b22e8a78a0a401fa113d164da37de6c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:10.397453 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c89492bb-103d-4bb2-a5b1-8e8846065bb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b22e8a78a0a401fa113d164da37de6c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:10.405889 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c89492bb-103d-4bb2-a5b1-8e8846065bb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b22e8a78a0a401fa113d164da37de6c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:10.406681 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c89492bb-103d-4bb2-a5b1-8e8846065bb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b22e8a78a0a401fa113d164da37de6c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:10.464529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 948/1907] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:37:10 2022] DELETE /identity/v3/users/1b22e8a78a0a401fa113d164da37de6c => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:10.466612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c8512-bb7c-41e3-a7b3-257be39f88f8 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:10.467210 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c8512-bb7c-41e3-a7b3-257be39f88f8 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:10.467301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c8512-bb7c-41e3-a7b3-257be39f88f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:10.509413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-f50c8512-bb7c-41e3-a7b3-257be39f88f8 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:10.511060 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-f50c8512-bb7c-41e3-a7b3-257be39f88f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:10.633325 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 960/1908] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:10.642539 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89492bb-103d-4bb2-a5b1-8e8846065bb3 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:10.644083 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c89492bb-103d-4bb2-a5b1-8e8846065bb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:10.712735 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:10.744854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89492bb-103d-4bb2-a5b1-8e8846065bb3 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:10.750341 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee28fc6-8744-4ddc-9d02-93aecc1bc135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:10.751103 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee28fc6-8744-4ddc-9d02-93aecc1bc135 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:10.751614 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee28fc6-8744-4ddc-9d02-93aecc1bc135 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:10.752438 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee28fc6-8744-4ddc-9d02-93aecc1bc135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:10.772494 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee28fc6-8744-4ddc-9d02-93aecc1bc135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:10.774130 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee28fc6-8744-4ddc-9d02-93aecc1bc135 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:10.787139 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 949/1909] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:37:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:37:11.342400 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c8512-bb7c-41e3-a7b3-257be39f88f8 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:11.344042 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-f50c8512-bb7c-41e3-a7b3-257be39f88f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:11.366573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c8512-bb7c-41e3-a7b3-257be39f88f8 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:11.383546 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb48745-d183-4fba-a468-ed4609ee7a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:11.384666 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb48745-d183-4fba-a468-ed4609ee7a76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:11.385031 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb48745-d183-4fba-a468-ed4609ee7a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:11.385394 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb48745-d183-4fba-a468-ed4609ee7a76 admin admin] PATH_INFO: `/v3/projects/67afa2b402b64f188c5ab850608c1edf` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:11.386451 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb48745-d183-4fba-a468-ed4609ee7a76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67afa2b402b64f188c5ab850608c1edf)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:11.388777 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb48745-d183-4fba-a468-ed4609ee7a76 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:11.450530 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:11.528114 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-eeb48745-d183-4fba-a468-ed4609ee7a76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67afa2b402b64f188c5ab850608c1edf', 'request_id': 'req-eeb48745-d183-4fba-a468-ed4609ee7a76'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:11.540245 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-eeb48745-d183-4fba-a468-ed4609ee7a76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67afa2b402b64f188c5ab850608c1edf', 'request_id': 'req-eeb48745-d183-4fba-a468-ed4609ee7a76'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:11.568502 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-eeb48745-d183-4fba-a468-ed4609ee7a76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67afa2b402b64f188c5ab850608c1edf', 'request_id': 'req-eeb48745-d183-4fba-a468-ed4609ee7a76'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:11.592230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-eeb48745-d183-4fba-a468-ed4609ee7a76 admin admin] The token cache is being invalidate because project 67afa2b402b64f188c5ab850608c1edf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:37:11.592760 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-eeb48745-d183-4fba-a468-ed4609ee7a76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:11.593869 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-eeb48745-d183-4fba-a468-ed4609ee7a76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:11.616598 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 961/1910] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:37:11 2022] DELETE /identity/v3/projects/67afa2b402b64f188c5ab850608c1edf => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:11.622444 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b30aa-5064-4fac-b9f1-03e746a414fe None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:11.623680 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b30aa-5064-4fac-b9f1-03e746a414fe None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:11.625171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b30aa-5064-4fac-b9f1-03e746a414fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:11.650284 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-3c4b30aa-5064-4fac-b9f1-03e746a414fe None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:11.651838 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3c4b30aa-5064-4fac-b9f1-03e746a414fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:11.710928 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb48745-d183-4fba-a468-ed4609ee7a76 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:11.713122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-eeb48745-d183-4fba-a468-ed4609ee7a76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:11.804629 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 950/1911] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:11.814730 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308091d-1c6f-4c76-84f9-85deace49a52 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:11.815100 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308091d-1c6f-4c76-84f9-85deace49a52 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:11.816492 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308091d-1c6f-4c76-84f9-85deace49a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:11.841424 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e308091d-1c6f-4c76-84f9-85deace49a52 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:11.842745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e308091d-1c6f-4c76-84f9-85deace49a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:11.852252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb48745-d183-4fba-a468-ed4609ee7a76 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:11.856396 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 951/1912] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:11.858372 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb39cf3-66b5-4628-868f-d21807a5e99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:11.859033 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb39cf3-66b5-4628-868f-d21807a5e99c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:11.859447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb39cf3-66b5-4628-868f-d21807a5e99c admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:11.859935 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb39cf3-66b5-4628-868f-d21807a5e99c admin admin] PATH_INFO: `/v3/users/65d857cf186442bdb0f8fd39ea2957dc` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:11.861103 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308091d-1c6f-4c76-84f9-85deace49a52 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:11.862479 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb39cf3-66b5-4628-868f-d21807a5e99c admin admin] RBAC: Authorizing `identity:delete_user(user_id=65d857cf186442bdb0f8fd39ea2957dc)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:11.865027 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb39cf3-66b5-4628-868f-d21807a5e99c admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:11.872461 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308091d-1c6f-4c76-84f9-85deace49a52 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:11.879829 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36321c98-4b33-4c5d-8eb6-c063f6a2f0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:11.880532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36321c98-4b33-4c5d-8eb6-c063f6a2f0ee admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:11.880903 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36321c98-4b33-4c5d-8eb6-c063f6a2f0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:11.881268 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36321c98-4b33-4c5d-8eb6-c063f6a2f0ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:11.882123 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36321c98-4b33-4c5d-8eb6-c063f6a2f0ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:37:11.882501 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36321c98-4b33-4c5d-8eb6-c063f6a2f0ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:11.885512 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36321c98-4b33-4c5d-8eb6-c063f6a2f0ee admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:11.901656 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 952/1913] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:37:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:37:11.907769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36321c98-4b33-4c5d-8eb6-c063f6a2f0ee admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:11.920951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36321c98-4b33-4c5d-8eb6-c063f6a2f0ee admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:11.928308 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6cfae4-4655-4563-b85d-e11834e81c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:11.929606 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6cfae4-4655-4563-b85d-e11834e81c7b admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:11.929606 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6cfae4-4655-4563-b85d-e11834e81c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:11.929881 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6cfae4-4655-4563-b85d-e11834e81c7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:11.930800 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6cfae4-4655-4563-b85d-e11834e81c7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:11.933363 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6cfae4-4655-4563-b85d-e11834e81c7b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:11.940484 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-2cb39cf3-66b5-4628-868f-d21807a5e99c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65d857cf186442bdb0f8fd39ea2957dc'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:11.941432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-2cb39cf3-66b5-4628-868f-d21807a5e99c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65d857cf186442bdb0f8fd39ea2957dc'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:11.956154 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-2cb39cf3-66b5-4628-868f-d21807a5e99c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65d857cf186442bdb0f8fd39ea2957dc'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:11.958170 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-2cb39cf3-66b5-4628-868f-d21807a5e99c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65d857cf186442bdb0f8fd39ea2957dc'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:11.968417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-2cb39cf3-66b5-4628-868f-d21807a5e99c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65d857cf186442bdb0f8fd39ea2957dc'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:11.983521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 953/1914] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:37:11 2022] POST /identity/v3/projects => generated 375 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:37:11.985388 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6cfae4-4655-4563-b85d-e11834e81c7b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:11.988793 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8c6cfae4-4655-4563-b85d-e11834e81c7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:12.004354 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 962/1915] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:37:11 2022] DELETE /identity/v3/users/65d857cf186442bdb0f8fd39ea2957dc => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:12.006813 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d3342-6391-438b-a175-b3fb90cbf9fb None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:12.007246 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d3342-6391-438b-a175-b3fb90cbf9fb None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:12.007603 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d3342-6391-438b-a175-b3fb90cbf9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:12.033211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-ef2d3342-6391-438b-a175-b3fb90cbf9fb None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:12.034816 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-ef2d3342-6391-438b-a175-b3fb90cbf9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:12.083231 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6cfae4-4655-4563-b85d-e11834e81c7b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:12.148799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:12.154911 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:12.203390 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc7feb-e0e1-460f-b4f0-53bfa3061449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:12.206079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc7feb-e0e1-460f-b4f0-53bfa3061449 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:12.207697 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc7feb-e0e1-460f-b4f0-53bfa3061449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:12.209007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc7feb-e0e1-460f-b4f0-53bfa3061449 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:12.210666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc7feb-e0e1-460f-b4f0-53bfa3061449 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:12.213474 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc7feb-e0e1-460f-b4f0-53bfa3061449 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:12.228910 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 963/1916] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:12.240565 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d3342-6391-438b-a175-b3fb90cbf9fb None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:12.242526 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-ef2d3342-6391-438b-a175-b3fb90cbf9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:12.270232 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 954/1917] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:37:11 2022] POST /identity/v3/users => generated 288 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:37:12.275377 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc7feb-e0e1-460f-b4f0-53bfa3061449 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:12.296698 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc7feb-e0e1-460f-b4f0-53bfa3061449 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:12.312288 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde2190-372a-4f2f-b39a-8ca008ce5221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:12.313022 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde2190-372a-4f2f-b39a-8ca008ce5221 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:12.313563 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde2190-372a-4f2f-b39a-8ca008ce5221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:12.314144 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde2190-372a-4f2f-b39a-8ca008ce5221 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:12.315113 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde2190-372a-4f2f-b39a-8ca008ce5221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:12.318501 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde2190-372a-4f2f-b39a-8ca008ce5221 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:12.348709 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 955/1918] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:37:12.352848 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde2190-372a-4f2f-b39a-8ca008ce5221 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:12.366666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde2190-372a-4f2f-b39a-8ca008ce5221 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:12.374798 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14881736-f45a-4992-b218-e8464e7a8e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:12.378436 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14881736-f45a-4992-b218-e8464e7a8e7d admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:12.379000 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14881736-f45a-4992-b218-e8464e7a8e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:12.379465 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14881736-f45a-4992-b218-e8464e7a8e7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:12.382167 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14881736-f45a-4992-b218-e8464e7a8e7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:12.385863 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14881736-f45a-4992-b218-e8464e7a8e7d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:12.393191 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d3342-6391-438b-a175-b3fb90cbf9fb None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:12.401530 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 956/1919] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:37:12.405250 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346aa9d-6b5d-46c7-83f2-e54353427f29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:12.405720 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14881736-f45a-4992-b218-e8464e7a8e7d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:12.406035 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346aa9d-6b5d-46c7-83f2-e54353427f29 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:12.406501 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346aa9d-6b5d-46c7-83f2-e54353427f29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:12.406815 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346aa9d-6b5d-46c7-83f2-e54353427f29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:12.425718 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14881736-f45a-4992-b218-e8464e7a8e7d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:12.433573 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c13bb-19a6-4c22-8653-2c9fb6d5a863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:12.433943 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346aa9d-6b5d-46c7-83f2-e54353427f29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:12.434678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c13bb-19a6-4c22-8653-2c9fb6d5a863 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:12.435042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c13bb-19a6-4c22-8653-2c9fb6d5a863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:12.435465 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c13bb-19a6-4c22-8653-2c9fb6d5a863 admin admin] PATH_INFO: `/v3/projects/85f88ec4713343179f6604b43af5719c/users/be1b75a397f249e492da050611b406aa/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:12.435733 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346aa9d-6b5d-46c7-83f2-e54353427f29 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:12.459708 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 964/1920] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:37:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:37:12.476818 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c13bb-19a6-4c22-8653-2c9fb6d5a863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85f88ec4713343179f6604b43af5719c, user_id=be1b75a397f249e492da050611b406aa, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:12.479802 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c13bb-19a6-4c22-8653-2c9fb6d5a863 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:12.517916 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 957/1921] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:37:12 2022] PUT /identity/v3/projects/85f88ec4713343179f6604b43af5719c/users/be1b75a397f249e492da050611b406aa/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:37:12.529717 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feca192-a43a-45a4-b908-6f1039e3fb51 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:12.530235 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feca192-a43a-45a4-b908-6f1039e3fb51 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:12.530235 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feca192-a43a-45a4-b908-6f1039e3fb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:12.570826 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-3feca192-a43a-45a4-b908-6f1039e3fb51 None None] MFA Rules not processed for user `be1b75a397f249e492da050611b406aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:12.575037 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-3feca192-a43a-45a4-b908-6f1039e3fb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:12.681750 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 965/1922] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:12.693463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c13bb-19a6-4c22-8653-2c9fb6d5a863 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:12.695254 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-972c13bb-19a6-4c22-8653-2c9fb6d5a863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:12.858059 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c13bb-19a6-4c22-8653-2c9fb6d5a863 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:12.878685 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c63a3f-89a3-4924-9794-4c874ebaa92b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:12.879458 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c63a3f-89a3-4924-9794-4c874ebaa92b service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:12.879853 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c63a3f-89a3-4924-9794-4c874ebaa92b service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:12.880203 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c63a3f-89a3-4924-9794-4c874ebaa92b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:12.891183 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c63a3f-89a3-4924-9794-4c874ebaa92b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:12.894013 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c63a3f-89a3-4924-9794-4c874ebaa92b service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:12.916922 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 958/1923] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:37:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:37:12.975272 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feca192-a43a-45a4-b908-6f1039e3fb51 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:12.976946 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-3feca192-a43a-45a4-b908-6f1039e3fb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:13.114217 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feca192-a43a-45a4-b908-6f1039e3fb51 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:13.125408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3e2f3-676b-4741-992b-f96184eb517d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:13.126126 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3e2f3-676b-4741-992b-f96184eb517d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:13.126412 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3e2f3-676b-4741-992b-f96184eb517d admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:13.126692 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3e2f3-676b-4741-992b-f96184eb517d admin admin] PATH_INFO: `/v3/projects/5a32b4a204ba4fbd9a3e2cd5551ad591` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:13.130391 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3e2f3-676b-4741-992b-f96184eb517d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a32b4a204ba4fbd9a3e2cd5551ad591)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:13.132459 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3e2f3-676b-4741-992b-f96184eb517d admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:13.151389 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:13.224904 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5aa3e2f3-676b-4741-992b-f96184eb517d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a32b4a204ba4fbd9a3e2cd5551ad591', 'request_id': 'req-5aa3e2f3-676b-4741-992b-f96184eb517d'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:13.233250 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5aa3e2f3-676b-4741-992b-f96184eb517d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a32b4a204ba4fbd9a3e2cd5551ad591', 'request_id': 'req-5aa3e2f3-676b-4741-992b-f96184eb517d'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:13.243578 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5aa3e2f3-676b-4741-992b-f96184eb517d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a32b4a204ba4fbd9a3e2cd5551ad591', 'request_id': 'req-5aa3e2f3-676b-4741-992b-f96184eb517d'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:13.264081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5aa3e2f3-676b-4741-992b-f96184eb517d admin admin] The token cache is being invalidate because project 5a32b4a204ba4fbd9a3e2cd5551ad591 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:37:13.264517 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5aa3e2f3-676b-4741-992b-f96184eb517d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:13.266021 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5aa3e2f3-676b-4741-992b-f96184eb517d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:13.268887 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 966/1924] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:37:12 2022] DELETE /identity/v3/projects/5a32b4a204ba4fbd9a3e2cd5551ad591 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:13.275779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ef240-373f-46b0-aa5a-11f47bdd7318 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:13.275779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ef240-373f-46b0-aa5a-11f47bdd7318 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:13.276152 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ef240-373f-46b0-aa5a-11f47bdd7318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:13.310901 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7d0ef240-373f-46b0-aa5a-11f47bdd7318 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:13.312899 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7d0ef240-373f-46b0-aa5a-11f47bdd7318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:13.443318 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 959/1925] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:13.451694 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af543b-d5cb-49ee-a3c7-16172b6c1ced None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:13.452078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af543b-d5cb-49ee-a3c7-16172b6c1ced None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:13.452397 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af543b-d5cb-49ee-a3c7-16172b6c1ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:13.478755 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-e2af543b-d5cb-49ee-a3c7-16172b6c1ced None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:13.480066 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-e2af543b-d5cb-49ee-a3c7-16172b6c1ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:13.495590 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ef240-373f-46b0-aa5a-11f47bdd7318 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:13.502846 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 967/1926] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:13.506794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ef240-373f-46b0-aa5a-11f47bdd7318 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:13.514276 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e2afb-995c-4201-a46a-525819b06b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:13.515168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e2afb-995c-4201-a46a-525819b06b94 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:13.515447 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e2afb-995c-4201-a46a-525819b06b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:13.515712 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963e2afb-995c-4201-a46a-525819b06b94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:13.516364 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963e2afb-995c-4201-a46a-525819b06b94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:37:13.517317 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963e2afb-995c-4201-a46a-525819b06b94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:13.519089 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963e2afb-995c-4201-a46a-525819b06b94 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:13.539806 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af543b-d5cb-49ee-a3c7-16172b6c1ced None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:13.545593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 960/1927] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:37:13.550121 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af543b-d5cb-49ee-a3c7-16172b6c1ced None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:13.555505 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99b539-a192-4958-84c5-5c5d176b1017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:13.556215 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99b539-a192-4958-84c5-5c5d176b1017 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:13.556606 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99b539-a192-4958-84c5-5c5d176b1017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:13.557003 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99b539-a192-4958-84c5-5c5d176b1017 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:13.557857 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99b539-a192-4958-84c5-5c5d176b1017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:13.559900 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99b539-a192-4958-84c5-5c5d176b1017 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:13.598924 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 968/1928] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:37:13 2022] POST /identity/v3/projects => generated 397 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:37:13.603837 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e2afb-995c-4201-a46a-525819b06b94 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:13.618436 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963e2afb-995c-4201-a46a-525819b06b94 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:13.634065 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:13.717190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac87f7-a529-4700-a05c-cdb8f67d2062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:13.718692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac87f7-a529-4700-a05c-cdb8f67d2062 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:13.719134 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac87f7-a529-4700-a05c-cdb8f67d2062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:13.719857 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac87f7-a529-4700-a05c-cdb8f67d2062 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:13.720773 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac87f7-a529-4700-a05c-cdb8f67d2062 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:13.723816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac87f7-a529-4700-a05c-cdb8f67d2062 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:13.755961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 961/1929] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:37:13 2022] POST /identity/v3/users => generated 299 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:37:13.758040 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99b539-a192-4958-84c5-5c5d176b1017 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:13.771804 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99b539-a192-4958-84c5-5c5d176b1017 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:13.777084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbc6de-e3ad-4812-b996-8ee6d80db841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:13.777708 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbc6de-e3ad-4812-b996-8ee6d80db841 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:13.777984 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbc6de-e3ad-4812-b996-8ee6d80db841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:13.778252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbc6de-e3ad-4812-b996-8ee6d80db841 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:13.778902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcbc6de-e3ad-4812-b996-8ee6d80db841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:13.780567 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcbc6de-e3ad-4812-b996-8ee6d80db841 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:13.792781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 969/1930] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:37:13.797559 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac87f7-a529-4700-a05c-cdb8f67d2062 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:13.812912 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac87f7-a529-4700-a05c-cdb8f67d2062 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:13.818125 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8223a61-d681-4788-a008-118d39f95c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:13.818825 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8223a61-d681-4788-a008-118d39f95c9a admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:13.819185 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8223a61-d681-4788-a008-118d39f95c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:13.819523 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8223a61-d681-4788-a008-118d39f95c9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:13.820252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8223a61-d681-4788-a008-118d39f95c9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:13.822164 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8223a61-d681-4788-a008-118d39f95c9a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:13.838151 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 962/1931] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:37:13.840021 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbc6de-e3ad-4812-b996-8ee6d80db841 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:13.851301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbc6de-e3ad-4812-b996-8ee6d80db841 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:13.856592 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f737363-457f-4089-a819-55db1184a731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:13.858083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f737363-457f-4089-a819-55db1184a731 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:13.858410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f737363-457f-4089-a819-55db1184a731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:13.858751 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f737363-457f-4089-a819-55db1184a731 admin admin] PATH_INFO: `/v3/projects/d2a9d535b8c1485996d177e58a12ef93/users/297899c3b75b4d1aa6914ad19b9437d3/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:13.884077 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f737363-457f-4089-a819-55db1184a731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2a9d535b8c1485996d177e58a12ef93, user_id=297899c3b75b4d1aa6914ad19b9437d3, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:13.887648 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f737363-457f-4089-a819-55db1184a731 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:13.906067 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 970/1932] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:37:13 2022] PUT /identity/v3/projects/d2a9d535b8c1485996d177e58a12ef93/users/297899c3b75b4d1aa6914ad19b9437d3/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:37:13.920090 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510be83-e5e6-4708-8679-5c4f6b327935 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:13.920544 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510be83-e5e6-4708-8679-5c4f6b327935 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:13.920922 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510be83-e5e6-4708-8679-5c4f6b327935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:13.983603 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0510be83-e5e6-4708-8679-5c4f6b327935 None None] MFA Rules not processed for user `297899c3b75b4d1aa6914ad19b9437d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:13.985045 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0510be83-e5e6-4708-8679-5c4f6b327935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:14.144987 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 963/1933] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:13 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:14.152851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f737363-457f-4089-a819-55db1184a731 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:14.154825 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-8f737363-457f-4089-a819-55db1184a731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:14.226337 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f737363-457f-4089-a819-55db1184a731 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:14.230853 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1905a19f-46d0-470e-b45f-4e286f6e355f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:14.231546 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905a19f-46d0-470e-b45f-4e286f6e355f service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:14.231885 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905a19f-46d0-470e-b45f-4e286f6e355f service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:14.232260 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905a19f-46d0-470e-b45f-4e286f6e355f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:14.243299 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1905a19f-46d0-470e-b45f-4e286f6e355f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:14.244852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1905a19f-46d0-470e-b45f-4e286f6e355f service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:14.259942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 971/1934] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:37:14 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:37:22.698795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510be83-e5e6-4708-8679-5c4f6b327935 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:22.781031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510be83-e5e6-4708-8679-5c4f6b327935 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:22.787536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464f8be-cbf7-452b-a668-bb14bc622560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:22.788183 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464f8be-cbf7-452b-a668-bb14bc622560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:22.788457 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464f8be-cbf7-452b-a668-bb14bc622560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:22.788732 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464f8be-cbf7-452b-a668-bb14bc622560 admin admin] PATH_INFO: `/v3/users/297899c3b75b4d1aa6914ad19b9437d3` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:22.789895 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9464f8be-cbf7-452b-a668-bb14bc622560 admin admin] RBAC: Authorizing `identity:delete_user(user_id=297899c3b75b4d1aa6914ad19b9437d3)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:22.791936 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9464f8be-cbf7-452b-a668-bb14bc622560 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:22.826196 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:22.853922 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9464f8be-cbf7-452b-a668-bb14bc622560 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '297899c3b75b4d1aa6914ad19b9437d3'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:22.854685 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9464f8be-cbf7-452b-a668-bb14bc622560 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '297899c3b75b4d1aa6914ad19b9437d3'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:22.865305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9464f8be-cbf7-452b-a668-bb14bc622560 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '297899c3b75b4d1aa6914ad19b9437d3'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:22.874383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9464f8be-cbf7-452b-a668-bb14bc622560 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '297899c3b75b4d1aa6914ad19b9437d3'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:22.875192 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9464f8be-cbf7-452b-a668-bb14bc622560 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '297899c3b75b4d1aa6914ad19b9437d3'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:22.900149 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 964/1935] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:37:22 2022] DELETE /identity/v3/users/297899c3b75b4d1aa6914ad19b9437d3 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:22.905254 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d690ab-345d-48c1-b14b-204240d9fe4b None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:22.905810 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d690ab-345d-48c1-b14b-204240d9fe4b None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:22.906237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d690ab-345d-48c1-b14b-204240d9fe4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:22.927642 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-06d690ab-345d-48c1-b14b-204240d9fe4b None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:22.928408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-06d690ab-345d-48c1-b14b-204240d9fe4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:22.967101 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:22.999195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 972/1936] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:23.010464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464f8be-cbf7-452b-a668-bb14bc622560 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:23.012028 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9464f8be-cbf7-452b-a668-bb14bc622560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:23.100925 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464f8be-cbf7-452b-a668-bb14bc622560 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:23.104882 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dd47d-85cc-41d6-a8b4-3cdb4c4d0126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:23.105532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dd47d-85cc-41d6-a8b4-3cdb4c4d0126 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:23.105768 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dd47d-85cc-41d6-a8b4-3cdb4c4d0126 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:23.106358 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dd47d-85cc-41d6-a8b4-3cdb4c4d0126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:23.120580 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22dd47d-85cc-41d6-a8b4-3cdb4c4d0126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:23.122305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22dd47d-85cc-41d6-a8b4-3cdb4c4d0126 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:23.134235 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 965/1937] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:37:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:37:23.409938 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d690ab-345d-48c1-b14b-204240d9fe4b None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:23.411429 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-06d690ab-345d-48c1-b14b-204240d9fe4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:23.427102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d690ab-345d-48c1-b14b-204240d9fe4b None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:23.431868 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f2a03-7070-42c0-acfb-3c3d67b544ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:23.432509 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f2a03-7070-42c0-acfb-3c3d67b544ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:23.432793 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f2a03-7070-42c0-acfb-3c3d67b544ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:23.433116 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f2a03-7070-42c0-acfb-3c3d67b544ec admin admin] PATH_INFO: `/v3/projects/d2a9d535b8c1485996d177e58a12ef93` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:23.434255 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f2a03-7070-42c0-acfb-3c3d67b544ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2a9d535b8c1485996d177e58a12ef93)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:23.436181 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f2a03-7070-42c0-acfb-3c3d67b544ec admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:23.483114 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-216f2a03-7070-42c0-acfb-3c3d67b544ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2a9d535b8c1485996d177e58a12ef93', 'request_id': 'req-216f2a03-7070-42c0-acfb-3c3d67b544ec'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:23.490797 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-216f2a03-7070-42c0-acfb-3c3d67b544ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2a9d535b8c1485996d177e58a12ef93', 'request_id': 'req-216f2a03-7070-42c0-acfb-3c3d67b544ec'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:23.500113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-216f2a03-7070-42c0-acfb-3c3d67b544ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2a9d535b8c1485996d177e58a12ef93', 'request_id': 'req-216f2a03-7070-42c0-acfb-3c3d67b544ec'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:23.513185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-216f2a03-7070-42c0-acfb-3c3d67b544ec admin admin] The token cache is being invalidate because project d2a9d535b8c1485996d177e58a12ef93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:37:23.513481 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-216f2a03-7070-42c0-acfb-3c3d67b544ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:23.514163 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-216f2a03-7070-42c0-acfb-3c3d67b544ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:23.515667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 973/1938] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:37:23 2022] DELETE /identity/v3/projects/d2a9d535b8c1485996d177e58a12ef93 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:26.153203 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dd47d-85cc-41d6-a8b4-3cdb4c4d0126 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:26.155047 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f22dd47d-85cc-41d6-a8b4-3cdb4c4d0126 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:26.169909 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dd47d-85cc-41d6-a8b4-3cdb4c4d0126 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:26.241146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738fdb7-d5cb-439d-bb46-7f554977a6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:26.241897 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738fdb7-d5cb-439d-bb46-7f554977a6de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:26.242623 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738fdb7-d5cb-439d-bb46-7f554977a6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:26.242979 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738fdb7-d5cb-439d-bb46-7f554977a6de admin admin] PATH_INFO: `/v3/users/be1b75a397f249e492da050611b406aa` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:26.244398 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f738fdb7-d5cb-439d-bb46-7f554977a6de admin admin] RBAC: Authorizing `identity:delete_user(user_id=be1b75a397f249e492da050611b406aa)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:26.246374 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f738fdb7-d5cb-439d-bb46-7f554977a6de admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:26.299002 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f738fdb7-d5cb-439d-bb46-7f554977a6de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be1b75a397f249e492da050611b406aa'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:26.299679 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f738fdb7-d5cb-439d-bb46-7f554977a6de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be1b75a397f249e492da050611b406aa'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:26.301096 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:26.309640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f738fdb7-d5cb-439d-bb46-7f554977a6de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be1b75a397f249e492da050611b406aa'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:26.318803 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f738fdb7-d5cb-439d-bb46-7f554977a6de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be1b75a397f249e492da050611b406aa'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:26.319971 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f738fdb7-d5cb-439d-bb46-7f554977a6de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be1b75a397f249e492da050611b406aa'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:26.338529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 966/1939] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:37:26 2022] DELETE /identity/v3/users/be1b75a397f249e492da050611b406aa => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:26.342837 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c205120-446d-46af-ac1f-a25b61753738 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:26.343308 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c205120-446d-46af-ac1f-a25b61753738 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:26.343400 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c205120-446d-46af-ac1f-a25b61753738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:26.364020 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-1c205120-446d-46af-ac1f-a25b61753738 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:26.365361 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-1c205120-446d-46af-ac1f-a25b61753738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:26.432976 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 974/1940] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:26.443182 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738fdb7-d5cb-439d-bb46-7f554977a6de admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:26.444564 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f738fdb7-d5cb-439d-bb46-7f554977a6de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:26.506891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738fdb7-d5cb-439d-bb46-7f554977a6de admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:26.510629 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d428fe-bf2e-40b1-966b-a52320490b80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:26.511242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d428fe-bf2e-40b1-966b-a52320490b80 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:26.511549 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d428fe-bf2e-40b1-966b-a52320490b80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:26.511820 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d428fe-bf2e-40b1-966b-a52320490b80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:26.521758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d428fe-bf2e-40b1-966b-a52320490b80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:26.523188 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d428fe-bf2e-40b1-966b-a52320490b80 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:26.533143 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 967/1941] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:37:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:37:26.802108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c205120-446d-46af-ac1f-a25b61753738 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:26.803568 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-1c205120-446d-46af-ac1f-a25b61753738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:26.816502 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c205120-446d-46af-ac1f-a25b61753738 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:26.821250 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa20368-9e1e-4ea1-8485-8e1255520601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:26.821894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa20368-9e1e-4ea1-8485-8e1255520601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:26.822177 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa20368-9e1e-4ea1-8485-8e1255520601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:26.822455 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa20368-9e1e-4ea1-8485-8e1255520601 admin admin] PATH_INFO: `/v3/projects/85f88ec4713343179f6604b43af5719c` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:26.823586 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa20368-9e1e-4ea1-8485-8e1255520601 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85f88ec4713343179f6604b43af5719c)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:26.825642 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa20368-9e1e-4ea1-8485-8e1255520601 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:26.875066 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ffa20368-9e1e-4ea1-8485-8e1255520601 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85f88ec4713343179f6604b43af5719c', 'request_id': 'req-ffa20368-9e1e-4ea1-8485-8e1255520601'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:26.882131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ffa20368-9e1e-4ea1-8485-8e1255520601 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85f88ec4713343179f6604b43af5719c', 'request_id': 'req-ffa20368-9e1e-4ea1-8485-8e1255520601'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:26.883790 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:26.889873 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ffa20368-9e1e-4ea1-8485-8e1255520601 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85f88ec4713343179f6604b43af5719c', 'request_id': 'req-ffa20368-9e1e-4ea1-8485-8e1255520601'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:26.900890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ffa20368-9e1e-4ea1-8485-8e1255520601 admin admin] The token cache is being invalidate because project 85f88ec4713343179f6604b43af5719c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:37:26.901185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ffa20368-9e1e-4ea1-8485-8e1255520601 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:26.901803 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ffa20368-9e1e-4ea1-8485-8e1255520601 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:26.903285 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 975/1942] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:37:26 2022] DELETE /identity/v3/projects/85f88ec4713343179f6604b43af5719c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:26.908616 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43010325-793d-459a-b090-c48640a1866a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:26.908919 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43010325-793d-459a-b090-c48640a1866a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:26.909191 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43010325-793d-459a-b090-c48640a1866a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:26.929223 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-43010325-793d-459a-b090-c48640a1866a None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:26.930462 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-43010325-793d-459a-b090-c48640a1866a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:26.994572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 968/1943] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:27.003107 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b4b48-6de3-460c-ae1e-8550cf9e3b1c None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:27.003392 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b4b48-6de3-460c-ae1e-8550cf9e3b1c None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:27.003693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b4b48-6de3-460c-ae1e-8550cf9e3b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:27.023427 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-c49b4b48-6de3-460c-ae1e-8550cf9e3b1c None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:27.024606 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-c49b4b48-6de3-460c-ae1e-8550cf9e3b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:27.033095 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 976/1944] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:27.037309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43010325-793d-459a-b090-c48640a1866a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:27.048644 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43010325-793d-459a-b090-c48640a1866a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:27.053890 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ecb554-6157-4dce-94b1-c3fbe9559587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:27.054517 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecb554-6157-4dce-94b1-c3fbe9559587 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:27.054795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecb554-6157-4dce-94b1-c3fbe9559587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:27.055070 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecb554-6157-4dce-94b1-c3fbe9559587 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:27.055724 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ecb554-6157-4dce-94b1-c3fbe9559587 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:37:27.056024 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ecb554-6157-4dce-94b1-c3fbe9559587 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:27.057668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ecb554-6157-4dce-94b1-c3fbe9559587 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:27.069801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 969/1945] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:37:27.074411 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b4b48-6de3-460c-ae1e-8550cf9e3b1c None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:27.084639 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b4b48-6de3-460c-ae1e-8550cf9e3b1c None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:27.089335 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639f886-4932-4b34-90cc-f0bc98248662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:27.089933 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639f886-4932-4b34-90cc-f0bc98248662 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:27.090232 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639f886-4932-4b34-90cc-f0bc98248662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:27.090490 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639f886-4932-4b34-90cc-f0bc98248662 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:27.091181 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639f886-4932-4b34-90cc-f0bc98248662 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:27.093118 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639f886-4932-4b34-90cc-f0bc98248662 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:27.124958 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 977/1946] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:37:27 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:37:27.126121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ecb554-6157-4dce-94b1-c3fbe9559587 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:27.136837 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ecb554-6157-4dce-94b1-c3fbe9559587 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:27.199557 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:27.208243 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe64b3-11d0-425c-8276-f186f1f96c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:27.209269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe64b3-11d0-425c-8276-f186f1f96c8a admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:27.209688 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe64b3-11d0-425c-8276-f186f1f96c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:27.210068 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe64b3-11d0-425c-8276-f186f1f96c8a admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:27.210992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbe64b3-11d0-425c-8276-f186f1f96c8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:27.213470 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbe64b3-11d0-425c-8276-f186f1f96c8a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:27.237492 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 970/1947] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:37:27 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:37:27.241945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639f886-4932-4b34-90cc-f0bc98248662 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:27.255943 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639f886-4932-4b34-90cc-f0bc98248662 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:27.261792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21e9d9-a9a2-4668-9dc5-7dbec2e96d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:27.261989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21e9d9-a9a2-4668-9dc5-7dbec2e96d89 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:27.262266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21e9d9-a9a2-4668-9dc5-7dbec2e96d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:27.262563 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21e9d9-a9a2-4668-9dc5-7dbec2e96d89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:27.263225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21e9d9-a9a2-4668-9dc5-7dbec2e96d89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:27.265044 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21e9d9-a9a2-4668-9dc5-7dbec2e96d89 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:27.284482 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 978/1948] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:37:27.285704 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe64b3-11d0-425c-8276-f186f1f96c8a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:27.298241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe64b3-11d0-425c-8276-f186f1f96c8a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:27.309269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63043d4a-9cac-4448-985b-6ebe27cad89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:27.310575 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63043d4a-9cac-4448-985b-6ebe27cad89c admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:27.310927 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63043d4a-9cac-4448-985b-6ebe27cad89c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:27.311267 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63043d4a-9cac-4448-985b-6ebe27cad89c admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:27.312534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63043d4a-9cac-4448-985b-6ebe27cad89c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:27.314665 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63043d4a-9cac-4448-985b-6ebe27cad89c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:27.326699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 971/1949] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:37:27.334215 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21e9d9-a9a2-4668-9dc5-7dbec2e96d89 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:27.345877 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21e9d9-a9a2-4668-9dc5-7dbec2e96d89 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:27.350630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9629f1-b4b8-4b13-83a1-f08b052adbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:27.351310 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9629f1-b4b8-4b13-83a1-f08b052adbbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:27.351592 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9629f1-b4b8-4b13-83a1-f08b052adbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:27.351865 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9629f1-b4b8-4b13-83a1-f08b052adbbf admin admin] PATH_INFO: `/v3/projects/32ef0737a3844bfe8f4bc14b09ebc88b/users/c9ac26a28e7b416b9786ddcacea5928e/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:27.369171 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9629f1-b4b8-4b13-83a1-f08b052adbbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=32ef0737a3844bfe8f4bc14b09ebc88b, user_id=c9ac26a28e7b416b9786ddcacea5928e, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:27.371893 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9629f1-b4b8-4b13-83a1-f08b052adbbf admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:27.395439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 979/1950] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:37:27 2022] PUT /identity/v3/projects/32ef0737a3844bfe8f4bc14b09ebc88b/users/c9ac26a28e7b416b9786ddcacea5928e/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:27.400711 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0dbd7-eb52-462c-96c7-b8ab418d5a25 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:27.401032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0dbd7-eb52-462c-96c7-b8ab418d5a25 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:27.401435 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0dbd7-eb52-462c-96c7-b8ab418d5a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:27.423883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c7b0dbd7-eb52-462c-96c7-b8ab418d5a25 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:27.425268 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c7b0dbd7-eb52-462c-96c7-b8ab418d5a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:27.499926 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 972/1951] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:37:27.511519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9629f1-b4b8-4b13-83a1-f08b052adbbf admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:27.513570 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-ce9629f1-b4b8-4b13-83a1-f08b052adbbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:27.603726 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9629f1-b4b8-4b13-83a1-f08b052adbbf admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:27.608451 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb12678-fdf2-4bda-8a50-f15685cddf9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:27.609396 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb12678-fdf2-4bda-8a50-f15685cddf9f service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:27.609688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb12678-fdf2-4bda-8a50-f15685cddf9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:27.610004 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb12678-fdf2-4bda-8a50-f15685cddf9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:27.620288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb12678-fdf2-4bda-8a50-f15685cddf9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:27.621862 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb12678-fdf2-4bda-8a50-f15685cddf9f service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:27.631897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 980/1952] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:37:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:37:32.368651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37592a4-7591-4a69-908b-f8fd4a90ac22 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:32.369218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37592a4-7591-4a69-908b-f8fd4a90ac22 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:32.369218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37592a4-7591-4a69-908b-f8fd4a90ac22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:32.404667 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c37592a4-7591-4a69-908b-f8fd4a90ac22 None None] MFA Rules not processed for user `c9ac26a28e7b416b9786ddcacea5928e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:32.405902 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c37592a4-7591-4a69-908b-f8fd4a90ac22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:32.473807 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 973/1953] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:32.478253 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb12678-fdf2-4bda-8a50-f15685cddf9f service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:32.488307 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb12678-fdf2-4bda-8a50-f15685cddf9f service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:32.493058 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74baf08c-88c3-4109-8d7d-34b83ef90877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:32.509121 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74baf08c-88c3-4109-8d7d-34b83ef90877 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:32.509417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74baf08c-88c3-4109-8d7d-34b83ef90877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:32.509701 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74baf08c-88c3-4109-8d7d-34b83ef90877 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:32.510384 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74baf08c-88c3-4109-8d7d-34b83ef90877 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:32.512343 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74baf08c-88c3-4109-8d7d-34b83ef90877 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:32.562436 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 981/1954] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:37:32 2022] POST /identity/v3/projects => generated 377 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:37:32.566648 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37592a4-7591-4a69-908b-f8fd4a90ac22 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:32.579870 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37592a4-7591-4a69-908b-f8fd4a90ac22 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:32.657373 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06fe92-eb91-4170-b7c1-a9d52f339843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:32.658597 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06fe92-eb91-4170-b7c1-a9d52f339843 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:32.658597 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06fe92-eb91-4170-b7c1-a9d52f339843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:32.658786 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06fe92-eb91-4170-b7c1-a9d52f339843 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:32.659284 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06fe92-eb91-4170-b7c1-a9d52f339843 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:32.661272 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06fe92-eb91-4170-b7c1-a9d52f339843 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:32.675612 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:32.696181 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 974/1955] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:37:32 2022] POST /identity/v3/users => generated 295 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:37:32.697623 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74baf08c-88c3-4109-8d7d-34b83ef90877 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:32.713525 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74baf08c-88c3-4109-8d7d-34b83ef90877 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:32.718529 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bacfc3-c9b9-4077-a661-08d6e4b395bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:32.719287 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bacfc3-c9b9-4077-a661-08d6e4b395bb admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:32.719991 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bacfc3-c9b9-4077-a661-08d6e4b395bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:32.720144 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bacfc3-c9b9-4077-a661-08d6e4b395bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:32.721033 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bacfc3-c9b9-4077-a661-08d6e4b395bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:32.725865 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bacfc3-c9b9-4077-a661-08d6e4b395bb admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:32.738219 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 982/1956] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:37:32.742093 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06fe92-eb91-4170-b7c1-a9d52f339843 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:32.753697 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06fe92-eb91-4170-b7c1-a9d52f339843 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:32.758755 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddcc912-7d48-4490-aa4d-491ed5750eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:32.759437 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcc912-7d48-4490-aa4d-491ed5750eed admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:32.760345 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcc912-7d48-4490-aa4d-491ed5750eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:32.760636 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcc912-7d48-4490-aa4d-491ed5750eed admin admin] PATH_INFO: `/v3/projects/111a918263d14198a3d24236431db365/users/87f47190e3234e0bb8cdb443e092e54f/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:32.778192 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddcc912-7d48-4490-aa4d-491ed5750eed admin admin] RBAC: Authorizing `identity:create_grant(project_id=111a918263d14198a3d24236431db365, user_id=87f47190e3234e0bb8cdb443e092e54f, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:32.781139 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddcc912-7d48-4490-aa4d-491ed5750eed admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:32.806351 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 975/1957] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:37:32 2022] PUT /identity/v3/projects/111a918263d14198a3d24236431db365/users/87f47190e3234e0bb8cdb443e092e54f/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:32.810331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bacfc3-c9b9-4077-a661-08d6e4b395bb admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:32.822072 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bacfc3-c9b9-4077-a661-08d6e4b395bb admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:32.869190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:32.887692 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8adab1-4c1a-4a3f-914f-a567c1111753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:32.888313 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8adab1-4c1a-4a3f-914f-a567c1111753 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:32.888623 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8adab1-4c1a-4a3f-914f-a567c1111753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:32.888920 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8adab1-4c1a-4a3f-914f-a567c1111753 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:32.889568 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8adab1-4c1a-4a3f-914f-a567c1111753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:32.891800 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8adab1-4c1a-4a3f-914f-a567c1111753 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:32.904897 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 983/1958] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:37:32.907947 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddcc912-7d48-4490-aa4d-491ed5750eed admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:32.919114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddcc912-7d48-4490-aa4d-491ed5750eed admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:32.924440 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc3df4-95f9-4bac-8bbc-3870564e6784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:32.925243 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc3df4-95f9-4bac-8bbc-3870564e6784 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:32.925527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc3df4-95f9-4bac-8bbc-3870564e6784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:32.925807 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc3df4-95f9-4bac-8bbc-3870564e6784 admin admin] PATH_INFO: `/v3/projects/111a918263d14198a3d24236431db365/users/87f47190e3234e0bb8cdb443e092e54f/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:32.927791 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc3df4-95f9-4bac-8bbc-3870564e6784 admin admin] RBAC: Authorizing `identity:create_grant(project_id=111a918263d14198a3d24236431db365, user_id=87f47190e3234e0bb8cdb443e092e54f, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:32.930485 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc3df4-95f9-4bac-8bbc-3870564e6784 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:32.955556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 976/1959] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:37:32 2022] PUT /identity/v3/projects/111a918263d14198a3d24236431db365/users/87f47190e3234e0bb8cdb443e092e54f/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:38.643544 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b97215-afb0-48ae-8fd8-f02d54610dd7 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:38.644064 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b97215-afb0-48ae-8fd8-f02d54610dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:38.644156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b97215-afb0-48ae-8fd8-f02d54610dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:38.685758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-92b97215-afb0-48ae-8fd8-f02d54610dd7 None None] MFA Rules not processed for user `144e9801ac1f470083d11647652ccdf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:38.691730 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-92b97215-afb0-48ae-8fd8-f02d54610dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:38.785740 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 984/1960] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:39.866046 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfadee0-a828-4cc9-b594-5066150ed5e1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:39.867004 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfadee0-a828-4cc9-b594-5066150ed5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:39.867149 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfadee0-a828-4cc9-b594-5066150ed5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:39.974059 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9dfadee0-a828-4cc9-b594-5066150ed5e1 None None] MFA Rules not processed for user `87f47190e3234e0bb8cdb443e092e54f`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:39.986617 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9dfadee0-a828-4cc9-b594-5066150ed5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:40.165367 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 977/1961] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:40.166162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b97215-afb0-48ae-8fd8-f02d54610dd7 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:40.169413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-92b97215-afb0-48ae-8fd8-f02d54610dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:40.277565 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b97215-afb0-48ae-8fd8-f02d54610dd7 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:40.285231 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c8ac9-897c-4ff3-aaa8-7ddbb666e0da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:40.285963 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c8ac9-897c-4ff3-aaa8-7ddbb666e0da service nova] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:40.286418 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c8ac9-897c-4ff3-aaa8-7ddbb666e0da service nova] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:40.286843 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c8ac9-897c-4ff3-aaa8-7ddbb666e0da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:40.377188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c8ac9-897c-4ff3-aaa8-7ddbb666e0da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:40.379731 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c8ac9-897c-4ff3-aaa8-7ddbb666e0da service nova] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:40.391187 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 985/1962] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:37:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:37:41.104014 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfadee0-a828-4cc9-b594-5066150ed5e1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:41.105563 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9dfadee0-a828-4cc9-b594-5066150ed5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:41.215395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfadee0-a828-4cc9-b594-5066150ed5e1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:41.221425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624793e-fc1e-4e2b-adac-9bd726196adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:41.222163 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624793e-fc1e-4e2b-adac-9bd726196adb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:41.222663 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624793e-fc1e-4e2b-adac-9bd726196adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:41.223016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624793e-fc1e-4e2b-adac-9bd726196adb admin admin] PATH_INFO: `/v3/users/2675e22c174344c4bacc13d24e425d82` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:41.224411 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5624793e-fc1e-4e2b-adac-9bd726196adb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2675e22c174344c4bacc13d24e425d82)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:41.226755 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5624793e-fc1e-4e2b-adac-9bd726196adb admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:41.338678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:INSERT=3,keystone:SELECT=91,keystone:DELETE=6 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:41.342848 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5624793e-fc1e-4e2b-adac-9bd726196adb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2675e22c174344c4bacc13d24e425d82'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:41.344549 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5624793e-fc1e-4e2b-adac-9bd726196adb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2675e22c174344c4bacc13d24e425d82'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:41.356843 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5624793e-fc1e-4e2b-adac-9bd726196adb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2675e22c174344c4bacc13d24e425d82'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:41.376143 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5624793e-fc1e-4e2b-adac-9bd726196adb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2675e22c174344c4bacc13d24e425d82'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:41.377154 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5624793e-fc1e-4e2b-adac-9bd726196adb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2675e22c174344c4bacc13d24e425d82'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:41.400795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 978/1963] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:37:41 2022] DELETE /identity/v3/users/2675e22c174344c4bacc13d24e425d82 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:41.405594 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe98d2-24be-48a6-8c8f-d0c279ac5f61 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:41.406147 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe98d2-24be-48a6-8c8f-d0c279ac5f61 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:41.406312 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe98d2-24be-48a6-8c8f-d0c279ac5f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:41.433625 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-62fe98d2-24be-48a6-8c8f-d0c279ac5f61 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:41.434922 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-62fe98d2-24be-48a6-8c8f-d0c279ac5f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:41.462139 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:41.598899 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 986/1964] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:41.609710 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624793e-fc1e-4e2b-adac-9bd726196adb admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:41.622235 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5624793e-fc1e-4e2b-adac-9bd726196adb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:41.834658 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624793e-fc1e-4e2b-adac-9bd726196adb admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:41.847028 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6650a-1010-4c1c-aa9e-0e24533e8e14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:41.847633 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6650a-1010-4c1c-aa9e-0e24533e8e14 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:41.847919 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6650a-1010-4c1c-aa9e-0e24533e8e14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:41.849353 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6650a-1010-4c1c-aa9e-0e24533e8e14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:41.873137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6650a-1010-4c1c-aa9e-0e24533e8e14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:41.880331 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6650a-1010-4c1c-aa9e-0e24533e8e14 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:41.923855 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 979/1965] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:37:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:37:42.731626 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe98d2-24be-48a6-8c8f-d0c279ac5f61 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:42.736888 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-62fe98d2-24be-48a6-8c8f-d0c279ac5f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:42.776655 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe98d2-24be-48a6-8c8f-d0c279ac5f61 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:42.806831 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df17f0b7-98f2-434b-8a3e-f5e909eb1f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:42.808379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17f0b7-98f2-434b-8a3e-f5e909eb1f00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:42.808998 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17f0b7-98f2-434b-8a3e-f5e909eb1f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:42.810140 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df17f0b7-98f2-434b-8a3e-f5e909eb1f00 admin admin] PATH_INFO: `/v3/projects/ed7726d1a1964225845ef2a25138001d` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:42.813950 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17f0b7-98f2-434b-8a3e-f5e909eb1f00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed7726d1a1964225845ef2a25138001d)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:42.816293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df17f0b7-98f2-434b-8a3e-f5e909eb1f00 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:42.979638 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-df17f0b7-98f2-434b-8a3e-f5e909eb1f00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed7726d1a1964225845ef2a25138001d', 'request_id': 'req-df17f0b7-98f2-434b-8a3e-f5e909eb1f00'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:42.995696 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-df17f0b7-98f2-434b-8a3e-f5e909eb1f00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed7726d1a1964225845ef2a25138001d', 'request_id': 'req-df17f0b7-98f2-434b-8a3e-f5e909eb1f00'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:43.008015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-df17f0b7-98f2-434b-8a3e-f5e909eb1f00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed7726d1a1964225845ef2a25138001d', 'request_id': 'req-df17f0b7-98f2-434b-8a3e-f5e909eb1f00'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:43.028228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-df17f0b7-98f2-434b-8a3e-f5e909eb1f00 admin admin] The token cache is being invalidate because project ed7726d1a1964225845ef2a25138001d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:37:43.028841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-df17f0b7-98f2-434b-8a3e-f5e909eb1f00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:43.029765 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-df17f0b7-98f2-434b-8a3e-f5e909eb1f00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:37:43.032953 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 987/1966] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:37:42 2022] DELETE /identity/v3/projects/ed7726d1a1964225845ef2a25138001d => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:43.039829 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4ac99-91c5-4817-8051-c093baa73cc7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:43.040203 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4ac99-91c5-4817-8051-c093baa73cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:43.040680 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4ac99-91c5-4817-8051-c093baa73cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:43.095017 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e1f4ac99-91c5-4817-8051-c093baa73cc7 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:43.098279 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e1f4ac99-91c5-4817-8051-c093baa73cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:43.241925 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 980/1967] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:43.250506 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5195d2-bf23-4fcd-981e-adedfb2b66cd None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:43.250967 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5195d2-bf23-4fcd-981e-adedfb2b66cd None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:43.251500 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5195d2-bf23-4fcd-981e-adedfb2b66cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:43.277871 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-9e5195d2-bf23-4fcd-981e-adedfb2b66cd None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:43.279575 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-9e5195d2-bf23-4fcd-981e-adedfb2b66cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:43.295988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4ac99-91c5-4817-8051-c093baa73cc7 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:43.309076 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 988/1968] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:43.312620 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4ac99-91c5-4817-8051-c093baa73cc7 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:43.320558 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd64760-17d5-4083-8685-9c01ff9d2c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:43.321479 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd64760-17d5-4083-8685-9c01ff9d2c6f admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:43.322819 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd64760-17d5-4083-8685-9c01ff9d2c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:43.322819 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd64760-17d5-4083-8685-9c01ff9d2c6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:43.334712 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd64760-17d5-4083-8685-9c01ff9d2c6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:37:43.335250 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd64760-17d5-4083-8685-9c01ff9d2c6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:43.339293 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd64760-17d5-4083-8685-9c01ff9d2c6f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:43.378783 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5195d2-bf23-4fcd-981e-adedfb2b66cd None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:43.388905 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5195d2-bf23-4fcd-981e-adedfb2b66cd None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:43.389429 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 981/1969] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:37:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:37:43.402852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72284be4-eb29-45f2-9a76-e9dc7f8674c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:43.403589 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72284be4-eb29-45f2-9a76-e9dc7f8674c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:43.404016 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72284be4-eb29-45f2-9a76-e9dc7f8674c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:43.404382 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72284be4-eb29-45f2-9a76-e9dc7f8674c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:43.405212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72284be4-eb29-45f2-9a76-e9dc7f8674c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:43.407962 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72284be4-eb29-45f2-9a76-e9dc7f8674c9 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:43.458539 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 989/1970] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:37:43 2022] POST /identity/v3/projects => generated 383 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:37:43.462676 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd64760-17d5-4083-8685-9c01ff9d2c6f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:43.483210 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd64760-17d5-4083-8685-9c01ff9d2c6f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:43.604774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:43.619050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e99188-f658-4a3d-8910-2f4891b77887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:43.619696 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e99188-f658-4a3d-8910-2f4891b77887 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:43.623979 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e99188-f658-4a3d-8910-2f4891b77887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:43.630832 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e99188-f658-4a3d-8910-2f4891b77887 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:43.631557 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e99188-f658-4a3d-8910-2f4891b77887 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:43.633903 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e99188-f658-4a3d-8910-2f4891b77887 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:43.669500 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 982/1971] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:37:43 2022] POST /identity/v3/users => generated 292 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:37:43.674320 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72284be4-eb29-45f2-9a76-e9dc7f8674c9 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:43.688041 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72284be4-eb29-45f2-9a76-e9dc7f8674c9 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:43.694969 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff869482-eb45-4dbb-b100-a14584649b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:43.696139 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff869482-eb45-4dbb-b100-a14584649b13 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:43.696685 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff869482-eb45-4dbb-b100-a14584649b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:43.696999 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff869482-eb45-4dbb-b100-a14584649b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:43.697737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff869482-eb45-4dbb-b100-a14584649b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:43.700028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff869482-eb45-4dbb-b100-a14584649b13 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:43.712759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 990/1972] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:37:43.716622 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e99188-f658-4a3d-8910-2f4891b77887 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:43.734854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e99188-f658-4a3d-8910-2f4891b77887 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:43.746363 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5064a6e-3393-491e-a1d4-b40dff45cad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:43.747007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5064a6e-3393-491e-a1d4-b40dff45cad7 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:43.747294 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5064a6e-3393-491e-a1d4-b40dff45cad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:43.747571 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5064a6e-3393-491e-a1d4-b40dff45cad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:43.749480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5064a6e-3393-491e-a1d4-b40dff45cad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:43.751982 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5064a6e-3393-491e-a1d4-b40dff45cad7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:43.784312 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff869482-eb45-4dbb-b100-a14584649b13 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:43.784942 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 983/1973] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:37:43.806379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff869482-eb45-4dbb-b100-a14584649b13 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:43.812782 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebd5ee-9cb2-4b7a-b4ed-685c78d3e969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:43.813955 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebd5ee-9cb2-4b7a-b4ed-685c78d3e969 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:43.814465 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebd5ee-9cb2-4b7a-b4ed-685c78d3e969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:43.814938 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebd5ee-9cb2-4b7a-b4ed-685c78d3e969 admin admin] PATH_INFO: `/v3/projects/c189090d654b46c880b0d40f1e46a563/users/41c8c7437dd9443d8ddf4d10d8da6758/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:43.842080 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebd5ee-9cb2-4b7a-b4ed-685c78d3e969 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c189090d654b46c880b0d40f1e46a563, user_id=41c8c7437dd9443d8ddf4d10d8da6758, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:43.846352 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebd5ee-9cb2-4b7a-b4ed-685c78d3e969 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:43.879821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 991/1974] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:37:43 2022] PUT /identity/v3/projects/c189090d654b46c880b0d40f1e46a563/users/41c8c7437dd9443d8ddf4d10d8da6758/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:37:43.884982 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807ba6a-6384-4115-94d0-61f4ab750669 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:43.885278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807ba6a-6384-4115-94d0-61f4ab750669 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:43.885558 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807ba6a-6384-4115-94d0-61f4ab750669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:43.909963 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b807ba6a-6384-4115-94d0-61f4ab750669 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:43.916214 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b807ba6a-6384-4115-94d0-61f4ab750669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:44.005497 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 984/1975] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:44.015834 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebd5ee-9cb2-4b7a-b4ed-685c78d3e969 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:44.017822 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-18ebd5ee-9cb2-4b7a-b4ed-685c78d3e969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:44.089663 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:44.100393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebd5ee-9cb2-4b7a-b4ed-685c78d3e969 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:44.106522 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9594d9-419a-471d-a078-ff719a642392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:44.107350 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9594d9-419a-471d-a078-ff719a642392 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:44.107734 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9594d9-419a-471d-a078-ff719a642392 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:44.108133 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9594d9-419a-471d-a078-ff719a642392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:44.127891 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9594d9-419a-471d-a078-ff719a642392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:44.129777 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9594d9-419a-471d-a078-ff719a642392 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:44.156684 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 992/1976] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:37:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:37:45.729012 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62b376-8735-4742-9e2e-d50ef9119eab None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:45.729012 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62b376-8735-4742-9e2e-d50ef9119eab None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:45.729692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62b376-8735-4742-9e2e-d50ef9119eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:45.765369 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1c62b376-8735-4742-9e2e-d50ef9119eab None None] MFA Rules not processed for user `41c8c7437dd9443d8ddf4d10d8da6758`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:45.767358 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1c62b376-8735-4742-9e2e-d50ef9119eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:45.850943 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 985/1977] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:45.865162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9594d9-419a-471d-a078-ff719a642392 service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:45.867882 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-8b9594d9-419a-471d-a078-ff719a642392 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:45.927804 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9594d9-419a-471d-a078-ff719a642392 service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:45.931403 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a012e0d-641a-492e-b6fc-59df8520cff4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:45.932013 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a012e0d-641a-492e-b6fc-59df8520cff4 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:45.932349 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a012e0d-641a-492e-b6fc-59df8520cff4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:45.932629 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a012e0d-641a-492e-b6fc-59df8520cff4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:45.941796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a012e0d-641a-492e-b6fc-59df8520cff4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:45.943240 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a012e0d-641a-492e-b6fc-59df8520cff4 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:45.952762 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 993/1978] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:37:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:37:47.350522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710da351-cf25-4919-a04a-d5ddb4e77329 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:47.351052 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710da351-cf25-4919-a04a-d5ddb4e77329 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:47.351052 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710da351-cf25-4919-a04a-d5ddb4e77329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:47.373870 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-710da351-cf25-4919-a04a-d5ddb4e77329 None None] MFA Rules not processed for user `41c8c7437dd9443d8ddf4d10d8da6758`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:47.375099 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-710da351-cf25-4919-a04a-d5ddb4e77329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:47.383204 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 986/1979] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:37:55.948443 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=31 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:56.294643 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f63501-9125-4140-b1f4-55513658c276 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:56.294982 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f63501-9125-4140-b1f4-55513658c276 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:56.295428 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f63501-9125-4140-b1f4-55513658c276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:56.316668 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-61f63501-9125-4140-b1f4-55513658c276 None None] MFA Rules not processed for user `85bb137d0fb74031a49a5c629b32c32b`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:56.318079 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-61f63501-9125-4140-b1f4-55513658c276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:56.373705 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 994/1980] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 20 13:37:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:37:56.448740 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710da351-cf25-4919-a04a-d5ddb4e77329 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:37:56.450285 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-710da351-cf25-4919-a04a-d5ddb4e77329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:56.512053 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:37:56.530353 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710da351-cf25-4919-a04a-d5ddb4e77329 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:37:56.536289 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae0360-720c-4993-b865-0e11b053e776 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9834211488354651803000c7e8fab80b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:37:56.536933 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae0360-720c-4993-b865-0e11b053e776 service swift] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:56.537226 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae0360-720c-4993-b865-0e11b053e776 service swift] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:56.537492 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae0360-720c-4993-b865-0e11b053e776 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:56.545693 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae0360-720c-4993-b865-0e11b053e776 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:37:56.548368 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae0360-720c-4993-b865-0e11b053e776 service swift] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:37:56.557121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 987/1981] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 20 13:37:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 13:37:59.248819 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187752cf-2464-446c-bcfc-06f442ea0fd5 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:37:59.249522 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187752cf-2464-446c-bcfc-06f442ea0fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:37:59.249522 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187752cf-2464-446c-bcfc-06f442ea0fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:37:59.269760 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-187752cf-2464-446c-bcfc-06f442ea0fd5 None None] MFA Rules not processed for user `c9ac26a28e7b416b9786ddcacea5928e`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:37:59.282972 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-187752cf-2464-446c-bcfc-06f442ea0fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:37:59.361127 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 995/1982] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:37:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:38:06.553210 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=8 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:38:09.353633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=50 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:38:12.539406 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736486d8-8f46-47bf-91c1-714e0ad965f1 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:38:12.540007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736486d8-8f46-47bf-91c1-714e0ad965f1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:38:12.541233 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736486d8-8f46-47bf-91c1-714e0ad965f1 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:38:12.542859 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 988/1983] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 20 13:38:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:38:12.549660 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c38e73-3b5e-4f5a-96a7-1917f66dc077 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:38:12.551605 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c38e73-3b5e-4f5a-96a7-1917f66dc077 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:38:12.551894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c38e73-3b5e-4f5a-96a7-1917f66dc077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:38:12.573249 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-c9c38e73-3b5e-4f5a-96a7-1917f66dc077 None None] MFA Rules not processed for user `ee5c2a3855cb4a668177aa19f337778d`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:38:12.574726 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-c9c38e73-3b5e-4f5a-96a7-1917f66dc077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:38:12.658616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 996/1984] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 20 13:38:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:38:12.670604 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736486d8-8f46-47bf-91c1-714e0ad965f1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:38:12.676994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-736486d8-8f46-47bf-91c1-714e0ad965f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:38:12.754955 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736486d8-8f46-47bf-91c1-714e0ad965f1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:38:12.760541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88a067-417a-4a92-a89e-d6be4d2838f9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87b5093e963c4753928a8c6091afc52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:38:12.761233 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88a067-417a-4a92-a89e-d6be4d2838f9 service glance] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:38:12.761507 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88a067-417a-4a92-a89e-d6be4d2838f9 service glance] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:38:12.761802 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88a067-417a-4a92-a89e-d6be4d2838f9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:38:12.773070 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f88a067-417a-4a92-a89e-d6be4d2838f9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:38:12.774518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f88a067-417a-4a92-a89e-d6be4d2838f9 service glance] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:38:12.785393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 989/1985] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:38:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:38:12.854812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e104716-5b3a-4af3-b2e3-ee8619cdcebc None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:38:12.855098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e104716-5b3a-4af3-b2e3-ee8619cdcebc None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:38:12.855397 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e104716-5b3a-4af3-b2e3-ee8619cdcebc None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:38:12.856764 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 997/1986] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 20 13:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:38:12.861922 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c162bdd-ad73-4063-9d53-59bade027e71 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:38:12.862246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c162bdd-ad73-4063-9d53-59bade027e71 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:38:12.862662 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c162bdd-ad73-4063-9d53-59bade027e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:38:12.885485 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-3c162bdd-ad73-4063-9d53-59bade027e71 None None] MFA Rules not processed for user `8a101adfc7a94bf1a4777350e9210a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:38:12.886690 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3c162bdd-ad73-4063-9d53-59bade027e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:38:12.893223 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 990/1987] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 20 13:38:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:38:12.904770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e104716-5b3a-4af3-b2e3-ee8619cdcebc None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:38:12.906448 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-9e104716-5b3a-4af3-b2e3-ee8619cdcebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:38:12.920032 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e104716-5b3a-4af3-b2e3-ee8619cdcebc None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:38:12.924266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d08cfd-f847-48fa-9a17-24aea3efded7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:38:12.924900 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d08cfd-f847-48fa-9a17-24aea3efded7 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:38:12.925192 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d08cfd-f847-48fa-9a17-24aea3efded7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:38:12.925500 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d08cfd-f847-48fa-9a17-24aea3efded7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:38:12.932718 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d08cfd-f847-48fa-9a17-24aea3efded7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:38:12.934193 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d08cfd-f847-48fa-9a17-24aea3efded7 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:38:12.942512 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 998/1988] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:38:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:38:13.026596 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093a2b0-3d00-484f-840b-8604563652d3 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:38:13.027025 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093a2b0-3d00-484f-840b-8604563652d3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:38:13.027390 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093a2b0-3d00-484f-840b-8604563652d3 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:38:13.029512 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 991/1989] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 20 13:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:38:13.034602 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d18ea7-9ff2-4e80-ae3b-42c9575238b4 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:38:13.034883 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d18ea7-9ff2-4e80-ae3b-42c9575238b4 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:38:13.035157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d18ea7-9ff2-4e80-ae3b-42c9575238b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:38:13.056538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-b9d18ea7-9ff2-4e80-ae3b-42c9575238b4 None None] MFA Rules not processed for user `ee5c2a3855cb4a668177aa19f337778d`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:38:13.057801 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b9d18ea7-9ff2-4e80-ae3b-42c9575238b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:38:13.068213 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 999/1990] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 20 13:38:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:38:13.078594 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093a2b0-3d00-484f-840b-8604563652d3 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:38:13.089476 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093a2b0-3d00-484f-840b-8604563652d3 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:38:13.093061 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457e8b8-50dc-4eb7-afb9-83f2208ce1f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:38:13.093654 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457e8b8-50dc-4eb7-afb9-83f2208ce1f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:38:13.093936 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457e8b8-50dc-4eb7-afb9-83f2208ce1f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:38:13.094209 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457e8b8-50dc-4eb7-afb9-83f2208ce1f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:38:13.105509 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457e8b8-50dc-4eb7-afb9-83f2208ce1f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:38:13.106939 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457e8b8-50dc-4eb7-afb9-83f2208ce1f2 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:38:13.120121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 992/1991] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:38:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:38:17.078030 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c73c2b-0069-49e2-807f-0d7ed7d6c910 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:38:17.078516 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c73c2b-0069-49e2-807f-0d7ed7d6c910 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:38:17.078629 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c73c2b-0069-49e2-807f-0d7ed7d6c910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:38:17.098974 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-f6c73c2b-0069-49e2-807f-0d7ed7d6c910 None None] MFA Rules not processed for user `85bb137d0fb74031a49a5c629b32c32b`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:38:17.100202 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-f6c73c2b-0069-49e2-807f-0d7ed7d6c910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:38:17.109081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1000/1992] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 20 13:38:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:38:17.123713 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457e8b8-50dc-4eb7-afb9-83f2208ce1f2 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:38:17.135671 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457e8b8-50dc-4eb7-afb9-83f2208ce1f2 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:38:17.142672 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18317d-b267-4971-bcfc-093adaf80d9d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9834211488354651803000c7e8fab80b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:38:17.143355 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18317d-b267-4971-bcfc-093adaf80d9d service swift] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:38:17.143636 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18317d-b267-4971-bcfc-093adaf80d9d service swift] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:38:17.143910 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18317d-b267-4971-bcfc-093adaf80d9d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:38:17.156916 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb18317d-b267-4971-bcfc-093adaf80d9d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:38:17.159051 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb18317d-b267-4971-bcfc-093adaf80d9d service swift] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:38:17.173051 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 993/1993] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 20 13:38:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 13:38:27.092136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=43 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:38:27.165414 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=50 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:38:32.986180 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c73c2b-0069-49e2-807f-0d7ed7d6c910 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:38:32.995698 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c73c2b-0069-49e2-807f-0d7ed7d6c910 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:38:32.999252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d292c-6f1f-4617-8f9f-96666dfea4c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:38:32.999862 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d292c-6f1f-4617-8f9f-96666dfea4c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:38:33.000384 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d292c-6f1f-4617-8f9f-96666dfea4c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:38:33.000657 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d292c-6f1f-4617-8f9f-96666dfea4c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:38:33.002507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-642d292c-6f1f-4617-8f9f-96666dfea4c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:38:33.064167 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d292c-6f1f-4617-8f9f-96666dfea4c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:38:33.065638 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d292c-6f1f-4617-8f9f-96666dfea4c8 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:38:33.075787 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1001/1994] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:38:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:38:43.071413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=30 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:38:50.831923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18317d-b267-4971-bcfc-093adaf80d9d service swift] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:38:50.834402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-bb18317d-b267-4971-bcfc-093adaf80d9d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:38:50.834764 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c557cf2-d046-4a77-a9cf-82ad352b3a5f None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:38:50.835151 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c557cf2-d046-4a77-a9cf-82ad352b3a5f None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:38:50.835384 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c557cf2-d046-4a77-a9cf-82ad352b3a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:38:50.868181 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-7c557cf2-d046-4a77-a9cf-82ad352b3a5f None None] MFA Rules not processed for user `85bb137d0fb74031a49a5c629b32c32b`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:38:50.869447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-7c557cf2-d046-4a77-a9cf-82ad352b3a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:38:50.879711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1002/1995] 10.222.0.23 () {58 vars in 967 bytes} [Thu Oct 20 13:38:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:38:50.895630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c557cf2-d046-4a77-a9cf-82ad352b3a5f None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:38:50.914581 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c557cf2-d046-4a77-a9cf-82ad352b3a5f None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:38:50.921409 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde89c91-0ed6-4e13-8be9-a2fefe4e631d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9834211488354651803000c7e8fab80b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:38:50.922323 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde89c91-0ed6-4e13-8be9-a2fefe4e631d service swift] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:38:50.922614 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde89c91-0ed6-4e13-8be9-a2fefe4e631d service swift] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:38:50.922894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde89c91-0ed6-4e13-8be9-a2fefe4e631d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:38:50.931959 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde89c91-0ed6-4e13-8be9-a2fefe4e631d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:38:50.933608 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde89c91-0ed6-4e13-8be9-a2fefe4e631d service swift] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:38:50.936774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18317d-b267-4971-bcfc-093adaf80d9d service swift] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:38:50.942842 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1003/1996] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 20 13:38:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 20 13:38:50.943758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19aeb20-0aff-4340-b5cf-bced5e3c9a55 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2440594934b64621a01c441133778155', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:38:50.944498 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19aeb20-0aff-4340-b5cf-bced5e3c9a55 service placement] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:38:50.944636 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19aeb20-0aff-4340-b5cf-bced5e3c9a55 service placement] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:38:50.944945 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19aeb20-0aff-4340-b5cf-bced5e3c9a55 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:38:50.946968 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f19aeb20-0aff-4340-b5cf-bced5e3c9a55 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:38:50.965473 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19aeb20-0aff-4340-b5cf-bced5e3c9a55 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:38:50.967561 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19aeb20-0aff-4340-b5cf-bced5e3c9a55 service placement] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:38:50.980022 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 994/1997] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:38:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:38:51.572635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974dca6a-be85-413a-a197-fad9caef4a94 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:38:51.572967 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974dca6a-be85-413a-a197-fad9caef4a94 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:38:51.573243 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974dca6a-be85-413a-a197-fad9caef4a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:38:51.591815 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-974dca6a-be85-413a-a197-fad9caef4a94 None None] MFA Rules not processed for user `2dd45d3b097c46c28eb94c5e0bd2a7a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:38:51.593042 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-974dca6a-be85-413a-a197-fad9caef4a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:38:51.653127 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1004/1998] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:38:51 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:39:00.975941 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=32 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:39:01.640598 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=34 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:39:10.012318 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e7a63-cb68-403d-828b-5c389ba4e39c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:39:10.012925 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e7a63-cb68-403d-828b-5c389ba4e39c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:39:10.012925 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e7a63-cb68-403d-828b-5c389ba4e39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:39:10.032040 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-bb8e7a63-cb68-403d-828b-5c389ba4e39c None None] MFA Rules not processed for user `41c8c7437dd9443d8ddf4d10d8da6758`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:39:10.034247 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-bb8e7a63-cb68-403d-828b-5c389ba4e39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:39:10.041054 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 995/1999] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:39:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:39:20.027168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:39:28.385360 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e15fa7-b3d0-4b50-8e1f-3d6ade02d2fd None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:39:28.385910 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e15fa7-b3d0-4b50-8e1f-3d6ade02d2fd None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:39:28.385910 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e15fa7-b3d0-4b50-8e1f-3d6ade02d2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:39:28.404469 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-25e15fa7-b3d0-4b50-8e1f-3d6ade02d2fd None None] MFA Rules not processed for user `144e9801ac1f470083d11647652ccdf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:39:28.405691 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-25e15fa7-b3d0-4b50-8e1f-3d6ade02d2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:39:28.460720 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1005/2000] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:39:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:39:38.453304 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=25 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:40:07.497291 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae438aa7-9d36-484b-b9ad-1033967b02ad None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:07.498542 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae438aa7-9d36-484b-b9ad-1033967b02ad None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:07.498917 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae438aa7-9d36-484b-b9ad-1033967b02ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:07.522259 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-ae438aa7-9d36-484b-b9ad-1033967b02ad None None] MFA Rules not processed for user `55526ddbea014e6cbf7c079cb204059e`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:40:07.523612 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ae438aa7-9d36-484b-b9ad-1033967b02ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:40:07.584281 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 996/2001] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:40:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:40:08.716108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45af4d6-4f06-4467-9803-232339e3f0e3 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:08.716639 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45af4d6-4f06-4467-9803-232339e3f0e3 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:08.716639 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45af4d6-4f06-4467-9803-232339e3f0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:08.743619 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-d45af4d6-4f06-4467-9803-232339e3f0e3 None None] MFA Rules not processed for user `fe56d1d4278c43cd96a39f7f75f373b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:40:08.744843 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-d45af4d6-4f06-4467-9803-232339e3f0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:40:08.805289 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1006/2002] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:40:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:40:13.178692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae438aa7-9d36-484b-b9ad-1033967b02ad None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:40:13.195294 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae438aa7-9d36-484b-b9ad-1033967b02ad None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:40:13.200339 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926a666-0551-4096-96f4-e553eb402dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:40:13.201038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926a666-0551-4096-96f4-e553eb402dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:13.201326 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926a666-0551-4096-96f4-e553eb402dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:13.201647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926a666-0551-4096-96f4-e553eb402dfd admin admin] PATH_INFO: `/v3/users/41c8c7437dd9443d8ddf4d10d8da6758` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:13.202770 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926a666-0551-4096-96f4-e553eb402dfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=41c8c7437dd9443d8ddf4d10d8da6758)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:40:13.205321 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0926a666-0551-4096-96f4-e553eb402dfd admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:40:13.257305 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0926a666-0551-4096-96f4-e553eb402dfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41c8c7437dd9443d8ddf4d10d8da6758'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:40:13.257980 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0926a666-0551-4096-96f4-e553eb402dfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41c8c7437dd9443d8ddf4d10d8da6758'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:40:13.264892 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0926a666-0551-4096-96f4-e553eb402dfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41c8c7437dd9443d8ddf4d10d8da6758'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:40:13.272783 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0926a666-0551-4096-96f4-e553eb402dfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41c8c7437dd9443d8ddf4d10d8da6758'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:40:13.273467 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0926a666-0551-4096-96f4-e553eb402dfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41c8c7437dd9443d8ddf4d10d8da6758'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:40:13.297340 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 997/2003] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:40:13 2022] DELETE /identity/v3/users/41c8c7437dd9443d8ddf4d10d8da6758 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:40:13.629458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45af4d6-4f06-4467-9803-232339e3f0e3 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:40:13.651866 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45af4d6-4f06-4467-9803-232339e3f0e3 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:40:13.716400 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe03607-2971-4558-ba29-685067ed78d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:40:13.717087 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe03607-2971-4558-ba29-685067ed78d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:13.717373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe03607-2971-4558-ba29-685067ed78d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:13.717651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe03607-2971-4558-ba29-685067ed78d4 admin admin] PATH_INFO: `/v3/projects/c189090d654b46c880b0d40f1e46a563` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:13.718795 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe03607-2971-4558-ba29-685067ed78d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c189090d654b46c880b0d40f1e46a563)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:40:13.721438 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe03607-2971-4558-ba29-685067ed78d4 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:40:13.775141 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ebe03607-2971-4558-ba29-685067ed78d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c189090d654b46c880b0d40f1e46a563', 'request_id': 'req-ebe03607-2971-4558-ba29-685067ed78d4'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:40:13.784434 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ebe03607-2971-4558-ba29-685067ed78d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c189090d654b46c880b0d40f1e46a563', 'request_id': 'req-ebe03607-2971-4558-ba29-685067ed78d4'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:40:13.792785 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ebe03607-2971-4558-ba29-685067ed78d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c189090d654b46c880b0d40f1e46a563', 'request_id': 'req-ebe03607-2971-4558-ba29-685067ed78d4'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:40:13.803908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ebe03607-2971-4558-ba29-685067ed78d4 admin admin] The token cache is being invalidate because project c189090d654b46c880b0d40f1e46a563 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:40:13.804199 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ebe03607-2971-4558-ba29-685067ed78d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:40:13.804827 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ebe03607-2971-4558-ba29-685067ed78d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:40:13.806419 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1007/2004] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:40:13 2022] DELETE /identity/v3/projects/c189090d654b46c880b0d40f1e46a563 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:40:13.811617 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd435adb-5cda-4a86-9502-9d0ed86a2a7d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:13.811903 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd435adb-5cda-4a86-9502-9d0ed86a2a7d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:13.812174 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd435adb-5cda-4a86-9502-9d0ed86a2a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:13.834556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-fd435adb-5cda-4a86-9502-9d0ed86a2a7d None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:40:13.835787 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fd435adb-5cda-4a86-9502-9d0ed86a2a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:40:13.897434 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 998/2005] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:40:13.905073 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81494e3a-ae29-4d66-915c-3c8ca5c75950 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:13.905385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81494e3a-ae29-4d66-915c-3c8ca5c75950 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:13.905665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81494e3a-ae29-4d66-915c-3c8ca5c75950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:13.926500 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-81494e3a-ae29-4d66-915c-3c8ca5c75950 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:40:13.927715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-81494e3a-ae29-4d66-915c-3c8ca5c75950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:40:13.937538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1008/2006] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:40:13.942754 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd435adb-5cda-4a86-9502-9d0ed86a2a7d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:40:13.953065 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd435adb-5cda-4a86-9502-9d0ed86a2a7d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:40:13.958687 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7186aea-7871-4e1a-8685-8ce1e2c231bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:40:13.959509 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7186aea-7871-4e1a-8685-8ce1e2c231bf admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:13.959945 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7186aea-7871-4e1a-8685-8ce1e2c231bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:13.960372 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7186aea-7871-4e1a-8685-8ce1e2c231bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:13.961239 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7186aea-7871-4e1a-8685-8ce1e2c231bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:40:13.961699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7186aea-7871-4e1a-8685-8ce1e2c231bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:40:13.963479 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7186aea-7871-4e1a-8685-8ce1e2c231bf admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:40:13.977000 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 999/2007] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:40:13.983445 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81494e3a-ae29-4d66-915c-3c8ca5c75950 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:40:13.994106 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81494e3a-ae29-4d66-915c-3c8ca5c75950 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:40:14.001373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ef214-bd92-4fdf-a8d9-dea5da0e8e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:40:14.002102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ef214-bd92-4fdf-a8d9-dea5da0e8e22 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:14.002388 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ef214-bd92-4fdf-a8d9-dea5da0e8e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:14.002662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ef214-bd92-4fdf-a8d9-dea5da0e8e22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:14.003297 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ef214-bd92-4fdf-a8d9-dea5da0e8e22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:40:14.005284 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ef214-bd92-4fdf-a8d9-dea5da0e8e22 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:40:14.038098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1009/2008] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:40:13 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:40:14.042536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7186aea-7871-4e1a-8685-8ce1e2c231bf admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:40:14.050221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:40:14.054494 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7186aea-7871-4e1a-8685-8ce1e2c231bf admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:40:14.114434 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c0176-ad4e-4c5c-8f34-72d0619a436e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:40:14.115050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c0176-ad4e-4c5c-8f34-72d0619a436e admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:14.115335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c0176-ad4e-4c5c-8f34-72d0619a436e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:14.115612 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c0176-ad4e-4c5c-8f34-72d0619a436e admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:14.116246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c0176-ad4e-4c5c-8f34-72d0619a436e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:40:14.118240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c0176-ad4e-4c5c-8f34-72d0619a436e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:40:14.141255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1000/2009] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:40:14 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:40:14.146695 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ef214-bd92-4fdf-a8d9-dea5da0e8e22 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:40:14.158747 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ef214-bd92-4fdf-a8d9-dea5da0e8e22 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:40:14.164482 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee507a9-333c-4ebe-89ca-acb3746e6f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:40:14.165142 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee507a9-333c-4ebe-89ca-acb3746e6f28 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:14.165874 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee507a9-333c-4ebe-89ca-acb3746e6f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:14.166229 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee507a9-333c-4ebe-89ca-acb3746e6f28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:14.166907 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee507a9-333c-4ebe-89ca-acb3746e6f28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:40:14.169442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee507a9-333c-4ebe-89ca-acb3746e6f28 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:40:14.170316 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:40:14.181191 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1010/2010] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:40:14.185947 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c0176-ad4e-4c5c-8f34-72d0619a436e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:40:14.197440 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c0176-ad4e-4c5c-8f34-72d0619a436e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:40:14.202147 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daed99b-be17-4560-915e-18a475e9f884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:40:14.202750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daed99b-be17-4560-915e-18a475e9f884 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:14.203035 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daed99b-be17-4560-915e-18a475e9f884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:14.203311 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daed99b-be17-4560-915e-18a475e9f884 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:14.204051 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daed99b-be17-4560-915e-18a475e9f884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:40:14.205847 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daed99b-be17-4560-915e-18a475e9f884 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:40:14.218004 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1001/2011] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:40:14.222747 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee507a9-333c-4ebe-89ca-acb3746e6f28 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:40:14.232651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee507a9-333c-4ebe-89ca-acb3746e6f28 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:40:14.237812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71a995-720d-447d-be83-ca028b15cbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:40:14.238508 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71a995-720d-447d-be83-ca028b15cbf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:14.238791 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71a995-720d-447d-be83-ca028b15cbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:14.239067 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71a995-720d-447d-be83-ca028b15cbf6 admin admin] PATH_INFO: `/v3/projects/accdb851c01342d49228faa016ff430c/users/ce7ce255f03e4d999ff111ccf3a799d7/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:14.254903 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b71a995-720d-447d-be83-ca028b15cbf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=accdb851c01342d49228faa016ff430c, user_id=ce7ce255f03e4d999ff111ccf3a799d7, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:40:14.258892 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b71a995-720d-447d-be83-ca028b15cbf6 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:40:14.273562 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1011/2012] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:40:14 2022] PUT /identity/v3/projects/accdb851c01342d49228faa016ff430c/users/ce7ce255f03e4d999ff111ccf3a799d7/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:40:14.278555 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0b28b-df21-4321-abc0-33a71114dccc None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:14.278853 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0b28b-df21-4321-abc0-33a71114dccc None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:14.279108 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0b28b-df21-4321-abc0-33a71114dccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:14.299157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0ab0b28b-df21-4321-abc0-33a71114dccc None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:40:14.300371 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0ab0b28b-df21-4321-abc0-33a71114dccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:40:14.361005 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1002/2013] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:40:14.371091 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71a995-720d-447d-be83-ca028b15cbf6 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:40:14.373172 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-6b71a995-720d-447d-be83-ca028b15cbf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:40:14.434104 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71a995-720d-447d-be83-ca028b15cbf6 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:40:14.437851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323fbac-8337-42a5-906c-f203df17e3bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:40:14.438788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323fbac-8337-42a5-906c-f203df17e3bc service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:14.439152 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323fbac-8337-42a5-906c-f203df17e3bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:14.439519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323fbac-8337-42a5-906c-f203df17e3bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:14.451903 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8323fbac-8337-42a5-906c-f203df17e3bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:40:14.453332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8323fbac-8337-42a5-906c-f203df17e3bc service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:40:14.463734 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1012/2014] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:40:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:40:20.270380 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29be138-15cc-46aa-a0a9-17dd5ab96771 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:20.270904 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29be138-15cc-46aa-a0a9-17dd5ab96771 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:20.270904 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29be138-15cc-46aa-a0a9-17dd5ab96771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:20.307590 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e29be138-15cc-46aa-a0a9-17dd5ab96771 None None] MFA Rules not processed for user `ce7ce255f03e4d999ff111ccf3a799d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:40:20.309631 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e29be138-15cc-46aa-a0a9-17dd5ab96771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:40:20.379624 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1003/2015] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:40:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:40:20.384194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323fbac-8337-42a5-906c-f203df17e3bc service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:40:20.395474 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323fbac-8337-42a5-906c-f203df17e3bc service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:40:20.400441 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ac345-5586-43e6-931c-0c37717cde16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:40:20.401068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ac345-5586-43e6-931c-0c37717cde16 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:20.401346 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ac345-5586-43e6-931c-0c37717cde16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:20.401619 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ac345-5586-43e6-931c-0c37717cde16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:20.402280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ac345-5586-43e6-931c-0c37717cde16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:40:20.404182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ac345-5586-43e6-931c-0c37717cde16 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:40:20.432080 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1013/2016] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:40:20 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:40:20.435942 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29be138-15cc-46aa-a0a9-17dd5ab96771 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:40:20.448121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29be138-15cc-46aa-a0a9-17dd5ab96771 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:40:20.463326 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:40:20.515249 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a407a-0cbd-4f7b-9057-e3d21ec44c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:40:20.515950 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a407a-0cbd-4f7b-9057-e3d21ec44c26 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:20.516402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a407a-0cbd-4f7b-9057-e3d21ec44c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:20.516887 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a407a-0cbd-4f7b-9057-e3d21ec44c26 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:20.517630 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a407a-0cbd-4f7b-9057-e3d21ec44c26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:40:20.519577 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a407a-0cbd-4f7b-9057-e3d21ec44c26 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:40:20.544686 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1004/2017] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:40:20 2022] POST /identity/v3/users => generated 298 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:40:20.545870 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ac345-5586-43e6-931c-0c37717cde16 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:40:20.556667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ac345-5586-43e6-931c-0c37717cde16 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:40:20.561759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d59e6-da5b-47a0-93a3-a861109f49e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:40:20.562363 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d59e6-da5b-47a0-93a3-a861109f49e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:20.562672 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d59e6-da5b-47a0-93a3-a861109f49e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:20.563502 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d59e6-da5b-47a0-93a3-a861109f49e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:20.564145 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d59e6-da5b-47a0-93a3-a861109f49e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:40:20.565780 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d59e6-da5b-47a0-93a3-a861109f49e4 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:40:20.576801 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1014/2018] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:40:20.581223 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a407a-0cbd-4f7b-9057-e3d21ec44c26 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:40:20.591959 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a407a-0cbd-4f7b-9057-e3d21ec44c26 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:40:20.597227 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb095d-1dcf-440b-a82c-ee99cdd815e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:40:20.597960 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb095d-1dcf-440b-a82c-ee99cdd815e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:20.598248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb095d-1dcf-440b-a82c-ee99cdd815e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:20.598528 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb095d-1dcf-440b-a82c-ee99cdd815e6 admin admin] PATH_INFO: `/v3/projects/773ed98417064aa18010359755de111b/users/81b411dd56004e84abb03889b165b2ba/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:20.626077 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb095d-1dcf-440b-a82c-ee99cdd815e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=773ed98417064aa18010359755de111b, user_id=81b411dd56004e84abb03889b165b2ba, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:40:20.630411 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb095d-1dcf-440b-a82c-ee99cdd815e6 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:40:20.644561 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1005/2019] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:40:20 2022] PUT /identity/v3/projects/773ed98417064aa18010359755de111b/users/81b411dd56004e84abb03889b165b2ba/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:40:20.648594 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d59e6-da5b-47a0-93a3-a861109f49e4 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:40:20.659607 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d59e6-da5b-47a0-93a3-a861109f49e4 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:40:20.735753 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a29ac7-83a0-4f43-b36e-d66809820400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:40:20.736368 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a29ac7-83a0-4f43-b36e-d66809820400 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:20.736645 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a29ac7-83a0-4f43-b36e-d66809820400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:20.736951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a29ac7-83a0-4f43-b36e-d66809820400 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:20.737598 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a29ac7-83a0-4f43-b36e-d66809820400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:40:20.739215 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a29ac7-83a0-4f43-b36e-d66809820400 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:40:20.756217 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1015/2020] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:40:20.760164 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb095d-1dcf-440b-a82c-ee99cdd815e6 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:40:20.771718 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb095d-1dcf-440b-a82c-ee99cdd815e6 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:40:20.779082 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b88701-2476-40c7-9b0f-e87aa3700b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:40:20.779799 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b88701-2476-40c7-9b0f-e87aa3700b2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:20.780086 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b88701-2476-40c7-9b0f-e87aa3700b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:20.780361 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b88701-2476-40c7-9b0f-e87aa3700b2f admin admin] PATH_INFO: `/v3/projects/773ed98417064aa18010359755de111b/users/81b411dd56004e84abb03889b165b2ba/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:20.786540 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b88701-2476-40c7-9b0f-e87aa3700b2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=773ed98417064aa18010359755de111b, user_id=81b411dd56004e84abb03889b165b2ba, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:40:20.789214 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b88701-2476-40c7-9b0f-e87aa3700b2f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:40:20.802024 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1006/2021] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:40:20 2022] PUT /identity/v3/projects/773ed98417064aa18010359755de111b/users/81b411dd56004e84abb03889b165b2ba/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:40:26.671433 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed900ad6-6ff0-4bd8-95de-ce01eb2087bc None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:26.672561 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed900ad6-6ff0-4bd8-95de-ce01eb2087bc None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:26.673833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed900ad6-6ff0-4bd8-95de-ce01eb2087bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:26.710962 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-ed900ad6-6ff0-4bd8-95de-ce01eb2087bc None None] MFA Rules not processed for user `81b411dd56004e84abb03889b165b2ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:40:26.712318 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-ed900ad6-6ff0-4bd8-95de-ce01eb2087bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:40:26.758020 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:40:26.820515 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1016/2022] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:40:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:40:26.829832 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b88701-2476-40c7-9b0f-e87aa3700b2f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:40:26.831802 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-52b88701-2476-40c7-9b0f-e87aa3700b2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:40:26.902206 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b88701-2476-40c7-9b0f-e87aa3700b2f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:40:26.905823 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bb136-9917-4858-9ef3-c5b6d10a9ac3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87b5093e963c4753928a8c6091afc52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:40:26.907094 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bb136-9917-4858-9ef3-c5b6d10a9ac3 service glance] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:26.907400 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bb136-9917-4858-9ef3-c5b6d10a9ac3 service glance] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:26.907675 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bb136-9917-4858-9ef3-c5b6d10a9ac3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:26.972676 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3bb136-9917-4858-9ef3-c5b6d10a9ac3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:40:26.974276 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3bb136-9917-4858-9ef3-c5b6d10a9ac3 service glance] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:40:26.992636 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1007/2023] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:40:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:40:27.155116 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb0014-e5a6-43e2-ab2b-f834c3cbab23 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:27.155404 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb0014-e5a6-43e2-ab2b-f834c3cbab23 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:27.155676 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb0014-e5a6-43e2-ab2b-f834c3cbab23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:27.181464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-50bb0014-e5a6-43e2-ab2b-f834c3cbab23 None None] MFA Rules not processed for user `85bb137d0fb74031a49a5c629b32c32b`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:40:27.183090 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-50bb0014-e5a6-43e2-ab2b-f834c3cbab23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:40:27.260202 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1017/2024] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 20 13:40:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:40:27.350508 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bb136-9917-4858-9ef3-c5b6d10a9ac3 service glance] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:40:27.353350 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ca3bb136-9917-4858-9ef3-c5b6d10a9ac3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:40:27.407222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:40:27.487222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bb136-9917-4858-9ef3-c5b6d10a9ac3 service glance] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:40:27.494206 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543eb48-8ae5-41ed-800d-cf7f92d98526 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9834211488354651803000c7e8fab80b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:40:27.494911 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543eb48-8ae5-41ed-800d-cf7f92d98526 service swift] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:27.495268 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543eb48-8ae5-41ed-800d-cf7f92d98526 service swift] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:27.495617 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543eb48-8ae5-41ed-800d-cf7f92d98526 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:27.510696 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0543eb48-8ae5-41ed-800d-cf7f92d98526 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:40:27.512470 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0543eb48-8ae5-41ed-800d-cf7f92d98526 service swift] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:40:27.525494 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1008/2025] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 20 13:40:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 13:40:28.319762 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb0014-e5a6-43e2-ab2b-f834c3cbab23 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:40:28.415693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb0014-e5a6-43e2-ab2b-f834c3cbab23 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:40:28.422352 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dba7da-c5bc-444c-9ef9-4a950d91a378 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:40:28.423072 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dba7da-c5bc-444c-9ef9-4a950d91a378 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:28.423551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dba7da-c5bc-444c-9ef9-4a950d91a378 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:28.424522 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dba7da-c5bc-444c-9ef9-4a950d91a378 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:28.430087 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-70dba7da-c5bc-444c-9ef9-4a950d91a378 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:40:28.627352 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dba7da-c5bc-444c-9ef9-4a950d91a378 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:40:28.633989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dba7da-c5bc-444c-9ef9-4a950d91a378 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:40:28.662826 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1018/2026] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:40:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:40:28.834583 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bf572-4c67-436a-948a-38e5a1554542 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:28.835780 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bf572-4c67-436a-948a-38e5a1554542 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:28.836127 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bf572-4c67-436a-948a-38e5a1554542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:28.859946 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-699bf572-4c67-436a-948a-38e5a1554542 None None] MFA Rules not processed for user `2dd45d3b097c46c28eb94c5e0bd2a7a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:40:28.862053 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-699bf572-4c67-436a-948a-38e5a1554542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:40:28.943301 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1009/2027] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:40:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:40:29.007089 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dba7da-c5bc-444c-9ef9-4a950d91a378 service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:40:29.008789 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-70dba7da-c5bc-444c-9ef9-4a950d91a378 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:40:29.021909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:40:29.085990 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dba7da-c5bc-444c-9ef9-4a950d91a378 service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:40:29.090179 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930399db-b867-45b7-9385-9729ec4cf613 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:40:29.090809 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930399db-b867-45b7-9385-9729ec4cf613 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:29.091092 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930399db-b867-45b7-9385-9729ec4cf613 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:29.091394 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930399db-b867-45b7-9385-9729ec4cf613 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:29.093186 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-930399db-b867-45b7-9385-9729ec4cf613 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:40:29.112749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930399db-b867-45b7-9385-9729ec4cf613 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:40:29.114257 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930399db-b867-45b7-9385-9729ec4cf613 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:40:29.130071 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1019/2028] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:40:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:40:38.010993 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af8e57-66c3-48f5-937d-baa2ab670901 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:38.011516 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af8e57-66c3-48f5-937d-baa2ab670901 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:38.011516 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af8e57-66c3-48f5-937d-baa2ab670901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:38.030804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f6af8e57-66c3-48f5-937d-baa2ab670901 None None] MFA Rules not processed for user `ce7ce255f03e4d999ff111ccf3a799d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:40:38.031995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f6af8e57-66c3-48f5-937d-baa2ab670901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:40:38.039054 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1010/2029] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:40:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:40:39.124384 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=22 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:40:40.451892 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51de17-c141-4a0f-ac34-2f8396468abe None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:40.452476 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51de17-c141-4a0f-ac34-2f8396468abe None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:40.452476 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51de17-c141-4a0f-ac34-2f8396468abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:40.472925 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-8a51de17-c141-4a0f-ac34-2f8396468abe None None] MFA Rules not processed for user `85bb137d0fb74031a49a5c629b32c32b`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:40:40.474222 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-8a51de17-c141-4a0f-ac34-2f8396468abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:40:40.483787 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1020/2030] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 20 13:40:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:40:40.505806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af8e57-66c3-48f5-937d-baa2ab670901 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:40:40.519221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af8e57-66c3-48f5-937d-baa2ab670901 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:40:40.524846 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7986c4e2-800e-455c-bc0b-90c764243700 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9834211488354651803000c7e8fab80b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:40:40.525600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986c4e2-800e-455c-bc0b-90c764243700 service swift] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:40.525875 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986c4e2-800e-455c-bc0b-90c764243700 service swift] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:40.526304 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986c4e2-800e-455c-bc0b-90c764243700 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:40.535687 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7986c4e2-800e-455c-bc0b-90c764243700 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:40:40.537560 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7986c4e2-800e-455c-bc0b-90c764243700 service swift] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:40:40.554548 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1011/2031] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 20 13:40:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 13:40:41.712024 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51de17-c141-4a0f-ac34-2f8396468abe None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:40:41.714475 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-8a51de17-c141-4a0f-ac34-2f8396468abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:40:41.731308 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51de17-c141-4a0f-ac34-2f8396468abe None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:40:41.736952 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4bf092-6d98-4e63-bfb6-707b7d39d1d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:40:41.737552 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bf092-6d98-4e63-bfb6-707b7d39d1d0 service nova] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:41.737830 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bf092-6d98-4e63-bfb6-707b7d39d1d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:41.738104 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bf092-6d98-4e63-bfb6-707b7d39d1d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:41.739834 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-2d4bf092-6d98-4e63-bfb6-707b7d39d1d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:40:41.798986 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4bf092-6d98-4e63-bfb6-707b7d39d1d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:40:41.803070 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4bf092-6d98-4e63-bfb6-707b7d39d1d0 service nova] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:40:41.810092 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1021/2032] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:40:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:40:50.542661 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=57 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:40:51.805620 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=31 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:40:58.311793 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf6568-22c1-40ef-a0fd-6f67fb7d2615 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:58.312337 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf6568-22c1-40ef-a0fd-6f67fb7d2615 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:58.312709 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf6568-22c1-40ef-a0fd-6f67fb7d2615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:58.334969 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-6baf6568-22c1-40ef-a0fd-6f67fb7d2615 None None] MFA Rules not processed for user `144e9801ac1f470083d11647652ccdf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:40:58.336507 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6baf6568-22c1-40ef-a0fd-6f67fb7d2615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:40:58.343639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1012/2033] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:40:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:40:58.678090 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4bf092-6d98-4e63-bfb6-707b7d39d1d0 service nova] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:40:58.679505 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-2d4bf092-6d98-4e63-bfb6-707b7d39d1d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:40:58.691708 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4bf092-6d98-4e63-bfb6-707b7d39d1d0 service nova] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:40:58.695273 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbb5b3-1670-4f2e-ac44-9fbc9a661971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:40:58.695870 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbb5b3-1670-4f2e-ac44-9fbc9a661971 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:58.696152 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbb5b3-1670-4f2e-ac44-9fbc9a661971 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:58.696424 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbb5b3-1670-4f2e-ac44-9fbc9a661971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:58.698106 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-fddbb5b3-1670-4f2e-ac44-9fbc9a661971 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:40:58.707475 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddbb5b3-1670-4f2e-ac44-9fbc9a661971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:40:58.708915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddbb5b3-1670-4f2e-ac44-9fbc9a661971 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:40:58.716607 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1022/2034] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:40:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:40:59.030896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf6568-22c1-40ef-a0fd-6f67fb7d2615 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:40:59.032505 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6baf6568-22c1-40ef-a0fd-6f67fb7d2615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:40:59.107192 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf6568-22c1-40ef-a0fd-6f67fb7d2615 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:40:59.112332 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705498a3-ee20-4e91-accc-f278f41d96f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2440594934b64621a01c441133778155', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:40:59.113171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705498a3-ee20-4e91-accc-f278f41d96f3 service placement] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:40:59.113455 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705498a3-ee20-4e91-accc-f278f41d96f3 service placement] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:40:59.113723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705498a3-ee20-4e91-accc-f278f41d96f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:40:59.115475 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-705498a3-ee20-4e91-accc-f278f41d96f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:40:59.127316 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705498a3-ee20-4e91-accc-f278f41d96f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:40:59.130187 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705498a3-ee20-4e91-accc-f278f41d96f3 service placement] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:40:59.148040 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1013/2035] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:40:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:41:03.103510 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d14ef-b572-49f1-bf78-6697e72990aa None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:03.104065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d14ef-b572-49f1-bf78-6697e72990aa None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:03.104065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d14ef-b572-49f1-bf78-6697e72990aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:03.117592 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705498a3-ee20-4e91-accc-f278f41d96f3 service placement] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:03.126483 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-b42d14ef-b572-49f1-bf78-6697e72990aa None None] MFA Rules not processed for user `c9ac26a28e7b416b9786ddcacea5928e`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:41:03.127706 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b42d14ef-b572-49f1-bf78-6697e72990aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:03.128976 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705498a3-ee20-4e91-accc-f278f41d96f3 service placement] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:03.133114 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de5291-b888-4bb6-b8bc-5da4f3377353 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:03.133849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de5291-b888-4bb6-b8bc-5da4f3377353 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:03.134238 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de5291-b888-4bb6-b8bc-5da4f3377353 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:03.134571 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de5291-b888-4bb6-b8bc-5da4f3377353 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:03.136596 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c8de5291-b888-4bb6-b8bc-5da4f3377353 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:03.148300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de5291-b888-4bb6-b8bc-5da4f3377353 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:03.150993 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de5291-b888-4bb6-b8bc-5da4f3377353 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:03.160111 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1014/2036] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:41:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:41:03.197942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1023/2037] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:41:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:41:09.036405 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de5291-b888-4bb6-b8bc-5da4f3377353 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:09.049003 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de5291-b888-4bb6-b8bc-5da4f3377353 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:09.054362 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923553b3-3a52-4681-bab1-cdb996fa9d09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:09.054951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923553b3-3a52-4681-bab1-cdb996fa9d09 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:09.055228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923553b3-3a52-4681-bab1-cdb996fa9d09 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:09.055498 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923553b3-3a52-4681-bab1-cdb996fa9d09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:09.057216 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-923553b3-3a52-4681-bab1-cdb996fa9d09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:09.114971 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923553b3-3a52-4681-bab1-cdb996fa9d09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:09.116536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923553b3-3a52-4681-bab1-cdb996fa9d09 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:09.125746 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1015/2038] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:41:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:41:09.581157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d14ef-b572-49f1-bf78-6697e72990aa None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:09.594038 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d14ef-b572-49f1-bf78-6697e72990aa None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:09.600296 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c225be-a630-44e0-974f-baf4dab1e266 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:09.600951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c225be-a630-44e0-974f-baf4dab1e266 service nova] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:09.601229 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c225be-a630-44e0-974f-baf4dab1e266 service nova] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:09.601499 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c225be-a630-44e0-974f-baf4dab1e266 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:09.603385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-96c225be-a630-44e0-974f-baf4dab1e266 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:09.664057 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c225be-a630-44e0-974f-baf4dab1e266 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:09.665709 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c225be-a630-44e0-974f-baf4dab1e266 service nova] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:09.675250 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1024/2039] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:41:09 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:41:13.063057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923553b3-3a52-4681-bab1-cdb996fa9d09 service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:13.073117 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923553b3-3a52-4681-bab1-cdb996fa9d09 service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:13.076778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205d7e0-1d1c-4ab3-ac69-164ac2ca5d58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:13.077504 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205d7e0-1d1c-4ab3-ac69-164ac2ca5d58 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:13.077855 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205d7e0-1d1c-4ab3-ac69-164ac2ca5d58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:13.078196 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205d7e0-1d1c-4ab3-ac69-164ac2ca5d58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:13.088375 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205d7e0-1d1c-4ab3-ac69-164ac2ca5d58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:13.089833 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205d7e0-1d1c-4ab3-ac69-164ac2ca5d58 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:13.099962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1016/2040] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:41:13 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:41:13.575392 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c225be-a630-44e0-974f-baf4dab1e266 service nova] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:13.588958 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c225be-a630-44e0-974f-baf4dab1e266 service nova] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:13.600012 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47269675-bf90-401a-86ee-b7a2e0ed182e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:13.600723 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47269675-bf90-401a-86ee-b7a2e0ed182e service nova] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:13.601238 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47269675-bf90-401a-86ee-b7a2e0ed182e service nova] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:13.601682 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47269675-bf90-401a-86ee-b7a2e0ed182e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:13.603464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-47269675-bf90-401a-86ee-b7a2e0ed182e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:13.620338 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47269675-bf90-401a-86ee-b7a2e0ed182e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:13.622289 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47269675-bf90-401a-86ee-b7a2e0ed182e service nova] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:13.656598 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1025/2041] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:41:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:41:15.469086 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205d7e0-1d1c-4ab3-ac69-164ac2ca5d58 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:15.483003 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205d7e0-1d1c-4ab3-ac69-164ac2ca5d58 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:15.490003 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5212ca-81cc-4ffa-a31e-59cb6b6bfd34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:15.490841 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5212ca-81cc-4ffa-a31e-59cb6b6bfd34 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:15.491333 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5212ca-81cc-4ffa-a31e-59cb6b6bfd34 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:15.491423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5212ca-81cc-4ffa-a31e-59cb6b6bfd34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:15.493546 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8d5212ca-81cc-4ffa-a31e-59cb6b6bfd34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:15.515094 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5212ca-81cc-4ffa-a31e-59cb6b6bfd34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:15.516677 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5212ca-81cc-4ffa-a31e-59cb6b6bfd34 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:15.530899 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1017/2042] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:41:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:41:23.628635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=65 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:41:25.526053 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=81 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:41:26.893259 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a24c86-c1cf-4096-a96a-80e4a4617e34 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:26.894466 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a24c86-c1cf-4096-a96a-80e4a4617e34 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:26.894833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a24c86-c1cf-4096-a96a-80e4a4617e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:26.915701 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-20a24c86-c1cf-4096-a96a-80e4a4617e34 None None] MFA Rules not processed for user `85bb137d0fb74031a49a5c629b32c32b`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:41:26.916976 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-20a24c86-c1cf-4096-a96a-80e4a4617e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:26.924442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1026/2043] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 20 13:41:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:41:26.938973 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5212ca-81cc-4ffa-a31e-59cb6b6bfd34 service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:26.952082 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5212ca-81cc-4ffa-a31e-59cb6b6bfd34 service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:26.959303 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168467d4-aef0-4b45-b6d0-7ab9fee563dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9834211488354651803000c7e8fab80b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:26.960150 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168467d4-aef0-4b45-b6d0-7ab9fee563dd service swift] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:26.960439 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168467d4-aef0-4b45-b6d0-7ab9fee563dd service swift] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:26.960720 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168467d4-aef0-4b45-b6d0-7ab9fee563dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:26.969727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168467d4-aef0-4b45-b6d0-7ab9fee563dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:26.971444 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168467d4-aef0-4b45-b6d0-7ab9fee563dd service swift] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:26.980259 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1018/2044] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 20 13:41:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 13:41:29.903124 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a24c86-c1cf-4096-a96a-80e4a4617e34 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:29.915774 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a24c86-c1cf-4096-a96a-80e4a4617e34 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:29.921217 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785ecd8-5bd8-45eb-8ab0-f807679cd926 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:29.921820 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785ecd8-5bd8-45eb-8ab0-f807679cd926 service nova] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:29.922102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785ecd8-5bd8-45eb-8ab0-f807679cd926 service nova] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:29.922376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785ecd8-5bd8-45eb-8ab0-f807679cd926 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:29.924100 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-7785ecd8-5bd8-45eb-8ab0-f807679cd926 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:29.938120 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7785ecd8-5bd8-45eb-8ab0-f807679cd926 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:29.940726 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7785ecd8-5bd8-45eb-8ab0-f807679cd926 service nova] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:29.951887 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1027/2045] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:41:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:41:36.976712 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=6 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:41:39.947386 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=9 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:41:48.027452 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168467d4-aef0-4b45-b6d0-7ab9fee563dd service swift] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:48.029555 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-168467d4-aef0-4b45-b6d0-7ab9fee563dd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:48.112242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168467d4-aef0-4b45-b6d0-7ab9fee563dd service swift] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:48.120715 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a1ab8-85c7-4058-94a7-d27f4780955b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:48.121590 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a1ab8-85c7-4058-94a7-d27f4780955b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:48.121883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a1ab8-85c7-4058-94a7-d27f4780955b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:48.122159 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a1ab8-85c7-4058-94a7-d27f4780955b admin admin] PATH_INFO: `/v3/users/2dd45d3b097c46c28eb94c5e0bd2a7a6` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:48.123917 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a1ab8-85c7-4058-94a7-d27f4780955b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dd45d3b097c46c28eb94c5e0bd2a7a6)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:48.126181 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a1ab8-85c7-4058-94a7-d27f4780955b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:48.191006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-d75a1ab8-85c7-4058-94a7-d27f4780955b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dd45d3b097c46c28eb94c5e0bd2a7a6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:48.192704 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-d75a1ab8-85c7-4058-94a7-d27f4780955b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dd45d3b097c46c28eb94c5e0bd2a7a6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:48.202588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-d75a1ab8-85c7-4058-94a7-d27f4780955b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dd45d3b097c46c28eb94c5e0bd2a7a6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:48.224893 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-d75a1ab8-85c7-4058-94a7-d27f4780955b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dd45d3b097c46c28eb94c5e0bd2a7a6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:48.226692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-d75a1ab8-85c7-4058-94a7-d27f4780955b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dd45d3b097c46c28eb94c5e0bd2a7a6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:48.257450 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1019/2046] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:41:48 2022] DELETE /identity/v3/users/2dd45d3b097c46c28eb94c5e0bd2a7a6 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:41:48.458110 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d49503-c86d-48d9-8f23-4176cb2d8ad8 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:48.459669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d49503-c86d-48d9-8f23-4176cb2d8ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:48.460972 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d49503-c86d-48d9-8f23-4176cb2d8ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:48.486677 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-e7d49503-c86d-48d9-8f23-4176cb2d8ad8 None None] MFA Rules not processed for user `85bb137d0fb74031a49a5c629b32c32b`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:41:48.488272 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-e7d49503-c86d-48d9-8f23-4176cb2d8ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:48.577371 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a1ab8-85c7-4058-94a7-d27f4780955b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:48.578941 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d75a1ab8-85c7-4058-94a7-d27f4780955b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:48.610380 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1028/2047] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 20 13:41:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:41:48.618366 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d49503-c86d-48d9-8f23-4176cb2d8ad8 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:48.620498 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-e7d49503-c86d-48d9-8f23-4176cb2d8ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:48.689961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a1ab8-85c7-4058-94a7-d27f4780955b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:48.696709 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30003c11-6b9a-4023-95c4-6c8a09eedf52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:48.697330 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30003c11-6b9a-4023-95c4-6c8a09eedf52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:48.697616 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30003c11-6b9a-4023-95c4-6c8a09eedf52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:48.697890 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30003c11-6b9a-4023-95c4-6c8a09eedf52 admin admin] PATH_INFO: `/v3/projects/f36e9fb394944eb0a0c2c0075dbc937c` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:48.699175 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30003c11-6b9a-4023-95c4-6c8a09eedf52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f36e9fb394944eb0a0c2c0075dbc937c)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:48.701271 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30003c11-6b9a-4023-95c4-6c8a09eedf52 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:48.728338 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:41:48.771139 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-30003c11-6b9a-4023-95c4-6c8a09eedf52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f36e9fb394944eb0a0c2c0075dbc937c', 'request_id': 'req-30003c11-6b9a-4023-95c4-6c8a09eedf52'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:48.780588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-30003c11-6b9a-4023-95c4-6c8a09eedf52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f36e9fb394944eb0a0c2c0075dbc937c', 'request_id': 'req-30003c11-6b9a-4023-95c4-6c8a09eedf52'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:48.784290 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d49503-c86d-48d9-8f23-4176cb2d8ad8 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:48.791445 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-30003c11-6b9a-4023-95c4-6c8a09eedf52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f36e9fb394944eb0a0c2c0075dbc937c', 'request_id': 'req-30003c11-6b9a-4023-95c4-6c8a09eedf52'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:48.804356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-30003c11-6b9a-4023-95c4-6c8a09eedf52 admin admin] The token cache is being invalidate because project f36e9fb394944eb0a0c2c0075dbc937c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:41:48.805012 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-30003c11-6b9a-4023-95c4-6c8a09eedf52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:48.805497 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-30003c11-6b9a-4023-95c4-6c8a09eedf52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:48.807336 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1020/2048] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:41:48 2022] DELETE /identity/v3/projects/f36e9fb394944eb0a0c2c0075dbc937c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:41:48.811920 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea604b-4705-47dd-a4a9-c43ce84aa692 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:48.812278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea604b-4705-47dd-a4a9-c43ce84aa692 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:48.812634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea604b-4705-47dd-a4a9-c43ce84aa692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:48.833258 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-7cea604b-4705-47dd-a4a9-c43ce84aa692 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:41:48.834501 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-7cea604b-4705-47dd-a4a9-c43ce84aa692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:48.859927 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3a17b-0062-40bd-96e3-fb4a77cd8672 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9834211488354651803000c7e8fab80b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:48.860553 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3a17b-0062-40bd-96e3-fb4a77cd8672 service swift] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:48.860838 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3a17b-0062-40bd-96e3-fb4a77cd8672 service swift] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:48.861145 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3a17b-0062-40bd-96e3-fb4a77cd8672 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:48.863389 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-07e3a17b-0062-40bd-96e3-fb4a77cd8672 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:48.902578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1021/2049] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:41:48.911784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b9e61-145c-4535-b855-1ac1e4217c0c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:48.912073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b9e61-145c-4535-b855-1ac1e4217c0c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:48.912340 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b9e61-145c-4535-b855-1ac1e4217c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:48.920696 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:41:48.933894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3a17b-0062-40bd-96e3-fb4a77cd8672 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:48.934767 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1c2b9e61-145c-4535-b855-1ac1e4217c0c None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:41:48.935939 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3a17b-0062-40bd-96e3-fb4a77cd8672 service swift] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:48.936087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1c2b9e61-145c-4535-b855-1ac1e4217c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:48.945603 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1022/2050] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:41:48.946785 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1029/2051] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 20 13:41:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 20 13:41:48.949768 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b9e61-145c-4535-b855-1ac1e4217c0c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:48.962501 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b9e61-145c-4535-b855-1ac1e4217c0c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:48.967575 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd75c3b-0714-4f2d-b1aa-759d8666d38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:48.968310 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd75c3b-0714-4f2d-b1aa-759d8666d38e admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:48.968781 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd75c3b-0714-4f2d-b1aa-759d8666d38e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:48.969196 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd75c3b-0714-4f2d-b1aa-759d8666d38e admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:48.970226 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd75c3b-0714-4f2d-b1aa-759d8666d38e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:41:48.970573 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd75c3b-0714-4f2d-b1aa-759d8666d38e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:48.972318 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd75c3b-0714-4f2d-b1aa-759d8666d38e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:48.991055 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1023/2052] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:41:48.992153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3a17b-0062-40bd-96e3-fb4a77cd8672 service swift] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:49.003938 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3a17b-0062-40bd-96e3-fb4a77cd8672 service swift] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:49.008911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ccf13-482a-4d9a-8217-e21bc531711e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:49.009523 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ccf13-482a-4d9a-8217-e21bc531711e admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:49.009807 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ccf13-482a-4d9a-8217-e21bc531711e admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:49.010080 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ccf13-482a-4d9a-8217-e21bc531711e admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:49.010721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ccf13-482a-4d9a-8217-e21bc531711e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:49.012651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ccf13-482a-4d9a-8217-e21bc531711e admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:49.062271 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1030/2053] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:41:48 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:41:49.063516 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd75c3b-0714-4f2d-b1aa-759d8666d38e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:49.080934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd75c3b-0714-4f2d-b1aa-759d8666d38e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:49.199516 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0368bc3-eaa6-41c6-a571-3ec493214dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:49.200074 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0368bc3-eaa6-41c6-a571-3ec493214dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:49.200363 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0368bc3-eaa6-41c6-a571-3ec493214dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:49.200697 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0368bc3-eaa6-41c6-a571-3ec493214dd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:49.201527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0368bc3-eaa6-41c6-a571-3ec493214dd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:49.203453 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0368bc3-eaa6-41c6-a571-3ec493214dd0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:49.237110 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1024/2054] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:41:49 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:41:49.241401 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ccf13-482a-4d9a-8217-e21bc531711e admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:49.273535 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ccf13-482a-4d9a-8217-e21bc531711e admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:49.287905 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc342c9c-369c-48ec-8840-8d9667a786b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:49.288416 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc342c9c-369c-48ec-8840-8d9667a786b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:49.288745 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc342c9c-369c-48ec-8840-8d9667a786b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:49.288984 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc342c9c-369c-48ec-8840-8d9667a786b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:49.289811 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc342c9c-369c-48ec-8840-8d9667a786b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:49.291659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc342c9c-369c-48ec-8840-8d9667a786b8 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:49.309264 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1031/2055] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:41:49.313171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0368bc3-eaa6-41c6-a571-3ec493214dd0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:49.327012 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0368bc3-eaa6-41c6-a571-3ec493214dd0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:49.333955 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22295276-292c-45de-b543-60159872c410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:49.334911 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22295276-292c-45de-b543-60159872c410 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:49.335476 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22295276-292c-45de-b543-60159872c410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:49.335953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22295276-292c-45de-b543-60159872c410 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:49.337172 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22295276-292c-45de-b543-60159872c410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:49.339129 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22295276-292c-45de-b543-60159872c410 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:49.360572 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1025/2056] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:41:49.365857 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc342c9c-369c-48ec-8840-8d9667a786b8 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:49.378479 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc342c9c-369c-48ec-8840-8d9667a786b8 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:49.384458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdaf6a-a6d4-4ad3-9dd3-0adef0d98928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:49.385551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdaf6a-a6d4-4ad3-9dd3-0adef0d98928 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:49.385974 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdaf6a-a6d4-4ad3-9dd3-0adef0d98928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:49.386254 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdaf6a-a6d4-4ad3-9dd3-0adef0d98928 admin admin] PATH_INFO: `/v3/projects/b4c1d1f324574edfbef5c1c067c814b3/users/f87ad83a919a47a8828dfdc7aeffea46/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:49.409708 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbdaf6a-a6d4-4ad3-9dd3-0adef0d98928 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4c1d1f324574edfbef5c1c067c814b3, user_id=f87ad83a919a47a8828dfdc7aeffea46, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:49.413504 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbdaf6a-a6d4-4ad3-9dd3-0adef0d98928 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:49.445181 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1032/2057] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:41:49 2022] PUT /identity/v3/projects/b4c1d1f324574edfbef5c1c067c814b3/users/f87ad83a919a47a8828dfdc7aeffea46/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:41:49.447480 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefc79c-c83a-495e-9795-baad87d7fb9d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:49.449904 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefc79c-c83a-495e-9795-baad87d7fb9d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:49.450376 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefc79c-c83a-495e-9795-baad87d7fb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:49.481257 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:41:49.513903 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9cefc79c-c83a-495e-9795-baad87d7fb9d None None] MFA Rules not processed for user `f87ad83a919a47a8828dfdc7aeffea46`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:41:49.518117 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9cefc79c-c83a-495e-9795-baad87d7fb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:49.654647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1026/2058] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:41:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:41:49.659082 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdaf6a-a6d4-4ad3-9dd3-0adef0d98928 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:49.780990 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdaf6a-a6d4-4ad3-9dd3-0adef0d98928 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:49.795779 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64411083-89a5-4916-a482-2e500ebaa4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:49.796404 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64411083-89a5-4916-a482-2e500ebaa4c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:49.796693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64411083-89a5-4916-a482-2e500ebaa4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:49.796981 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64411083-89a5-4916-a482-2e500ebaa4c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:49.797676 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64411083-89a5-4916-a482-2e500ebaa4c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:49.799546 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64411083-89a5-4916-a482-2e500ebaa4c8 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:49.854936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1033/2059] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:41:49 2022] POST /identity/v3/projects => generated 373 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:41:49.858936 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cefc79c-c83a-495e-9795-baad87d7fb9d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:49.874442 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cefc79c-c83a-495e-9795-baad87d7fb9d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:49.997559 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed197705-d5dc-4448-98d9-433163321684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:49.998321 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed197705-d5dc-4448-98d9-433163321684 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:49.998603 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed197705-d5dc-4448-98d9-433163321684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:49.998884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed197705-d5dc-4448-98d9-433163321684 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:49.999540 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed197705-d5dc-4448-98d9-433163321684 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:50.001559 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed197705-d5dc-4448-98d9-433163321684 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:50.031473 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1027/2060] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:41:49 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:41:50.034454 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64411083-89a5-4916-a482-2e500ebaa4c8 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:50.047177 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64411083-89a5-4916-a482-2e500ebaa4c8 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:50.053308 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f9953-1080-4904-ae66-8921f9f79e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:50.054197 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f9953-1080-4904-ae66-8921f9f79e4e admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:50.054486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f9953-1080-4904-ae66-8921f9f79e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:50.054838 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f9953-1080-4904-ae66-8921f9f79e4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:50.055748 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4f9953-1080-4904-ae66-8921f9f79e4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:50.057782 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4f9953-1080-4904-ae66-8921f9f79e4e admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:50.070858 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1034/2061] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:41:50.075690 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed197705-d5dc-4448-98d9-433163321684 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:50.086878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed197705-d5dc-4448-98d9-433163321684 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:50.097047 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd2771-6f65-4933-a61a-256f1561062a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:50.097735 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd2771-6f65-4933-a61a-256f1561062a admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:50.098644 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd2771-6f65-4933-a61a-256f1561062a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:50.098962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd2771-6f65-4933-a61a-256f1561062a admin admin] PATH_INFO: `/v3/projects/b824dbd6c748461988621d35e05d1d2f/users/efd1d28679934f79905365972f83432e/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:50.118330 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd2771-6f65-4933-a61a-256f1561062a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b824dbd6c748461988621d35e05d1d2f, user_id=efd1d28679934f79905365972f83432e, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:50.122985 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd2771-6f65-4933-a61a-256f1561062a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:50.145471 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1028/2062] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:41:50 2022] PUT /identity/v3/projects/b824dbd6c748461988621d35e05d1d2f/users/efd1d28679934f79905365972f83432e/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:41:50.150206 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f9953-1080-4904-ae66-8921f9f79e4e admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:50.161393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f9953-1080-4904-ae66-8921f9f79e4e admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:50.245411 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d72bd-e3ac-4891-8610-132fdd83266d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:50.246155 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d72bd-e3ac-4891-8610-132fdd83266d admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:50.246536 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d72bd-e3ac-4891-8610-132fdd83266d admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:50.246821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d72bd-e3ac-4891-8610-132fdd83266d admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:50.247462 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d72bd-e3ac-4891-8610-132fdd83266d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:50.249339 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d72bd-e3ac-4891-8610-132fdd83266d admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:50.265972 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1035/2063] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:41:50.270828 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd2771-6f65-4933-a61a-256f1561062a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:50.283370 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd2771-6f65-4933-a61a-256f1561062a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:50.289510 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93648c2-86df-48f8-8165-48aa2208a821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:50.290552 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93648c2-86df-48f8-8165-48aa2208a821 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:50.291106 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93648c2-86df-48f8-8165-48aa2208a821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:50.291551 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93648c2-86df-48f8-8165-48aa2208a821 admin admin] PATH_INFO: `/v3/projects/b824dbd6c748461988621d35e05d1d2f/users/efd1d28679934f79905365972f83432e/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:50.293634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93648c2-86df-48f8-8165-48aa2208a821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b824dbd6c748461988621d35e05d1d2f, user_id=efd1d28679934f79905365972f83432e, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:50.296507 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93648c2-86df-48f8-8165-48aa2208a821 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:50.309611 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1029/2064] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:41:50 2022] PUT /identity/v3/projects/b824dbd6c748461988621d35e05d1d2f/users/efd1d28679934f79905365972f83432e/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:41:50.332173 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c572f39-3827-457b-8e43-c783f5816915 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:50.332799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c572f39-3827-457b-8e43-c783f5816915 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:50.333185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c572f39-3827-457b-8e43-c783f5816915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:50.340568 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:41:50.367128 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-5c572f39-3827-457b-8e43-c783f5816915 None None] MFA Rules not processed for user `efd1d28679934f79905365972f83432e`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:41:50.368504 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-5c572f39-3827-457b-8e43-c783f5816915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:50.451240 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1036/2065] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:41:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:41:50.462812 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93648c2-86df-48f8-8165-48aa2208a821 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:50.464570 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a93648c2-86df-48f8-8165-48aa2208a821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:50.552535 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93648c2-86df-48f8-8165-48aa2208a821 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:50.562915 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0ff41-2fc2-4b29-b70b-a477e2bc8375 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:50.564004 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0ff41-2fc2-4b29-b70b-a477e2bc8375 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:50.564004 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0ff41-2fc2-4b29-b70b-a477e2bc8375 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:50.564327 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0ff41-2fc2-4b29-b70b-a477e2bc8375 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:50.572100 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:41:50.576652 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c0ff41-2fc2-4b29-b70b-a477e2bc8375 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:50.577874 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c0ff41-2fc2-4b29-b70b-a477e2bc8375 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:50.589925 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1030/2066] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:41:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:41:51.040740 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c572f39-3827-457b-8e43-c783f5816915 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:51.161307 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c572f39-3827-457b-8e43-c783f5816915 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:51.165603 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fddbd7-5b6f-4595-b6a5-25433716589a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:51.166251 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fddbd7-5b6f-4595-b6a5-25433716589a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:51.166540 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fddbd7-5b6f-4595-b6a5-25433716589a admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:51.166802 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fddbd7-5b6f-4595-b6a5-25433716589a admin admin] PATH_INFO: `/v3/users/f87ad83a919a47a8828dfdc7aeffea46` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:51.167936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fddbd7-5b6f-4595-b6a5-25433716589a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f87ad83a919a47a8828dfdc7aeffea46)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:51.169911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fddbd7-5b6f-4595-b6a5-25433716589a admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:51.231279 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-b2fddbd7-5b6f-4595-b6a5-25433716589a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f87ad83a919a47a8828dfdc7aeffea46'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:51.231787 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-b2fddbd7-5b6f-4595-b6a5-25433716589a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f87ad83a919a47a8828dfdc7aeffea46'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:51.247666 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-b2fddbd7-5b6f-4595-b6a5-25433716589a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f87ad83a919a47a8828dfdc7aeffea46'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:51.248445 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-b2fddbd7-5b6f-4595-b6a5-25433716589a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f87ad83a919a47a8828dfdc7aeffea46'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:51.256502 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-b2fddbd7-5b6f-4595-b6a5-25433716589a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f87ad83a919a47a8828dfdc7aeffea46'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:51.279350 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1037/2067] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:41:51 2022] DELETE /identity/v3/users/f87ad83a919a47a8828dfdc7aeffea46 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:41:51.282490 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaea280-4522-437d-822b-78273142f290 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:51.283011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaea280-4522-437d-822b-78273142f290 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:51.283011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaea280-4522-437d-822b-78273142f290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:51.305293 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-acaea280-4522-437d-822b-78273142f290 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:41:51.306495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-acaea280-4522-437d-822b-78273142f290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:51.380647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1031/2068] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:41:51.391712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fddbd7-5b6f-4595-b6a5-25433716589a admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:51.393343 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b2fddbd7-5b6f-4595-b6a5-25433716589a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:51.416141 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:41:51.499413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fddbd7-5b6f-4595-b6a5-25433716589a admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:51.513521 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05ccc0-f76f-4dfd-ba51-dbcd430bb1bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:51.514266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05ccc0-f76f-4dfd-ba51-dbcd430bb1bf service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:51.514593 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05ccc0-f76f-4dfd-ba51-dbcd430bb1bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:51.515317 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05ccc0-f76f-4dfd-ba51-dbcd430bb1bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:51.529615 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05ccc0-f76f-4dfd-ba51-dbcd430bb1bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:51.533441 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05ccc0-f76f-4dfd-ba51-dbcd430bb1bf service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:51.553535 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1038/2069] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:41:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:41:51.873789 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaea280-4522-437d-822b-78273142f290 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:51.875235 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-acaea280-4522-437d-822b-78273142f290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:51.888933 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaea280-4522-437d-822b-78273142f290 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:51.893650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38dc3be-e1fe-4ff2-b96c-64a2678291ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:51.894273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38dc3be-e1fe-4ff2-b96c-64a2678291ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:51.894556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38dc3be-e1fe-4ff2-b96c-64a2678291ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:51.895398 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38dc3be-e1fe-4ff2-b96c-64a2678291ec admin admin] PATH_INFO: `/v3/projects/b4c1d1f324574edfbef5c1c067c814b3` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:51.897194 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38dc3be-e1fe-4ff2-b96c-64a2678291ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4c1d1f324574edfbef5c1c067c814b3)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:51.899141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38dc3be-e1fe-4ff2-b96c-64a2678291ec admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:51.954639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a38dc3be-e1fe-4ff2-b96c-64a2678291ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4c1d1f324574edfbef5c1c067c814b3', 'request_id': 'req-a38dc3be-e1fe-4ff2-b96c-64a2678291ec'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:51.963644 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a38dc3be-e1fe-4ff2-b96c-64a2678291ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4c1d1f324574edfbef5c1c067c814b3', 'request_id': 'req-a38dc3be-e1fe-4ff2-b96c-64a2678291ec'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:51.975387 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a38dc3be-e1fe-4ff2-b96c-64a2678291ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4c1d1f324574edfbef5c1c067c814b3', 'request_id': 'req-a38dc3be-e1fe-4ff2-b96c-64a2678291ec'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:51.990189 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a38dc3be-e1fe-4ff2-b96c-64a2678291ec admin admin] The token cache is being invalidate because project b4c1d1f324574edfbef5c1c067c814b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:41:51.990598 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a38dc3be-e1fe-4ff2-b96c-64a2678291ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:51.991315 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a38dc3be-e1fe-4ff2-b96c-64a2678291ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:51.993145 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1032/2070] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:41:51 2022] DELETE /identity/v3/projects/b4c1d1f324574edfbef5c1c067c814b3 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:41:51.997508 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05ccc0-f76f-4dfd-ba51-dbcd430bb1bf service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:52.008633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05ccc0-f76f-4dfd-ba51-dbcd430bb1bf service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:52.073857 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863c2d2-cc77-49b4-9e48-4bb1be68a4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:52.074587 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863c2d2-cc77-49b4-9e48-4bb1be68a4a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:52.074781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863c2d2-cc77-49b4-9e48-4bb1be68a4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:52.075022 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863c2d2-cc77-49b4-9e48-4bb1be68a4a5 admin admin] PATH_INFO: `/v3/users/efd1d28679934f79905365972f83432e` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:52.076202 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2863c2d2-cc77-49b4-9e48-4bb1be68a4a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efd1d28679934f79905365972f83432e)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:52.078160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2863c2d2-cc77-49b4-9e48-4bb1be68a4a5 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:52.134965 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-2863c2d2-cc77-49b4-9e48-4bb1be68a4a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efd1d28679934f79905365972f83432e'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:52.135457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-2863c2d2-cc77-49b4-9e48-4bb1be68a4a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efd1d28679934f79905365972f83432e'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:52.151409 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-2863c2d2-cc77-49b4-9e48-4bb1be68a4a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efd1d28679934f79905365972f83432e'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:52.152115 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-2863c2d2-cc77-49b4-9e48-4bb1be68a4a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efd1d28679934f79905365972f83432e'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:52.159852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-2863c2d2-cc77-49b4-9e48-4bb1be68a4a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efd1d28679934f79905365972f83432e'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:52.182053 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1039/2071] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:41:51 2022] DELETE /identity/v3/users/efd1d28679934f79905365972f83432e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:41:52.563774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38dc3be-e1fe-4ff2-b96c-64a2678291ec admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:52.566254 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a38dc3be-e1fe-4ff2-b96c-64a2678291ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:52.588265 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38dc3be-e1fe-4ff2-b96c-64a2678291ec admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:52.701938 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53352563-e491-45e0-b5c7-41513dbbc976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:52.702595 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53352563-e491-45e0-b5c7-41513dbbc976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:52.702874 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53352563-e491-45e0-b5c7-41513dbbc976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:52.703144 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53352563-e491-45e0-b5c7-41513dbbc976 admin admin] PATH_INFO: `/v3/projects/b824dbd6c748461988621d35e05d1d2f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:52.704437 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53352563-e491-45e0-b5c7-41513dbbc976 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b824dbd6c748461988621d35e05d1d2f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:52.706514 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53352563-e491-45e0-b5c7-41513dbbc976 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:52.735400 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:41:52.787352 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-53352563-e491-45e0-b5c7-41513dbbc976 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b824dbd6c748461988621d35e05d1d2f', 'request_id': 'req-53352563-e491-45e0-b5c7-41513dbbc976'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:52.807875 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-53352563-e491-45e0-b5c7-41513dbbc976 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b824dbd6c748461988621d35e05d1d2f', 'request_id': 'req-53352563-e491-45e0-b5c7-41513dbbc976'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:52.823290 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-53352563-e491-45e0-b5c7-41513dbbc976 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b824dbd6c748461988621d35e05d1d2f', 'request_id': 'req-53352563-e491-45e0-b5c7-41513dbbc976'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:52.836518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-53352563-e491-45e0-b5c7-41513dbbc976 admin admin] The token cache is being invalidate because project b824dbd6c748461988621d35e05d1d2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:41:52.837062 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-53352563-e491-45e0-b5c7-41513dbbc976 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:52.837802 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-53352563-e491-45e0-b5c7-41513dbbc976 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:52.839715 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1033/2072] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:41:52 2022] DELETE /identity/v3/projects/b824dbd6c748461988621d35e05d1d2f => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:41:52.849933 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61014a6f-f128-439a-9d2a-00cc5b5c0080 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:52.849933 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61014a6f-f128-439a-9d2a-00cc5b5c0080 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:52.849933 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61014a6f-f128-439a-9d2a-00cc5b5c0080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:52.876335 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-61014a6f-f128-439a-9d2a-00cc5b5c0080 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:41:52.877760 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-61014a6f-f128-439a-9d2a-00cc5b5c0080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:52.906866 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:41:52.946625 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1040/2073] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:41:52.956462 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b7500-3a7a-4dcd-8579-c34d2c66be68 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:52.956752 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b7500-3a7a-4dcd-8579-c34d2c66be68 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:52.957102 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b7500-3a7a-4dcd-8579-c34d2c66be68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:52.977851 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-229b7500-3a7a-4dcd-8579-c34d2c66be68 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:41:52.979102 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-229b7500-3a7a-4dcd-8579-c34d2c66be68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:52.987643 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1034/2074] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:41:52.992012 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61014a6f-f128-439a-9d2a-00cc5b5c0080 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:53.002096 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61014a6f-f128-439a-9d2a-00cc5b5c0080 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:53.007290 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77534a7a-63bb-4422-a934-57c440fb4754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:53.007910 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77534a7a-63bb-4422-a934-57c440fb4754 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:53.008194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77534a7a-63bb-4422-a934-57c440fb4754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:53.008470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77534a7a-63bb-4422-a934-57c440fb4754 admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:53.009174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77534a7a-63bb-4422-a934-57c440fb4754 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:41:53.009482 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77534a7a-63bb-4422-a934-57c440fb4754 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:53.011111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77534a7a-63bb-4422-a934-57c440fb4754 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:53.027817 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1041/2075] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:41:53.028939 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b7500-3a7a-4dcd-8579-c34d2c66be68 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:53.038802 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b7500-3a7a-4dcd-8579-c34d2c66be68 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:53.043843 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f71206-3716-4743-a154-dff639aee059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:53.044534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f71206-3716-4743-a154-dff639aee059 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:53.044817 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f71206-3716-4743-a154-dff639aee059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:53.045130 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f71206-3716-4743-a154-dff639aee059 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:53.045957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f71206-3716-4743-a154-dff639aee059 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:53.048230 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f71206-3716-4743-a154-dff639aee059 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:53.079890 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1035/2076] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:41:53 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:41:53.084517 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77534a7a-63bb-4422-a934-57c440fb4754 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:53.094757 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77534a7a-63bb-4422-a934-57c440fb4754 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:53.161834 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e2bfb1-a3a9-40c4-9138-71a3c3e525d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:53.162469 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2bfb1-a3a9-40c4-9138-71a3c3e525d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:53.162750 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2bfb1-a3a9-40c4-9138-71a3c3e525d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:53.163027 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2bfb1-a3a9-40c4-9138-71a3c3e525d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:53.163667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e2bfb1-a3a9-40c4-9138-71a3c3e525d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:53.165665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e2bfb1-a3a9-40c4-9138-71a3c3e525d3 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:53.188785 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1042/2077] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:41:53 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:41:53.193174 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f71206-3716-4743-a154-dff639aee059 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:53.204065 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f71206-3716-4743-a154-dff639aee059 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:53.212574 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4038e6-62a7-45d0-b764-106485e667a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:53.213228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4038e6-62a7-45d0-b764-106485e667a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:53.213500 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4038e6-62a7-45d0-b764-106485e667a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:53.213780 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4038e6-62a7-45d0-b764-106485e667a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:53.214431 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4038e6-62a7-45d0-b764-106485e667a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:53.216032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4038e6-62a7-45d0-b764-106485e667a7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:53.230090 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1036/2078] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:41:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:41:53.231267 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e2bfb1-a3a9-40c4-9138-71a3c3e525d3 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:53.242517 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e2bfb1-a3a9-40c4-9138-71a3c3e525d3 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:53.247515 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2662ce-c41e-451b-bece-2cd1d153c250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:53.248112 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2662ce-c41e-451b-bece-2cd1d153c250 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:53.248387 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2662ce-c41e-451b-bece-2cd1d153c250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:53.248652 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2662ce-c41e-451b-bece-2cd1d153c250 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:53.249339 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2662ce-c41e-451b-bece-2cd1d153c250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:53.250947 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2662ce-c41e-451b-bece-2cd1d153c250 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:53.263685 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1043/2079] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:41:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:41:53.267989 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4038e6-62a7-45d0-b764-106485e667a7 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:53.278910 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4038e6-62a7-45d0-b764-106485e667a7 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:53.283718 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d3f57-6d19-4c0f-9080-ced4f751137f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:53.284460 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d3f57-6d19-4c0f-9080-ced4f751137f admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:53.284806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d3f57-6d19-4c0f-9080-ced4f751137f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:53.285172 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d3f57-6d19-4c0f-9080-ced4f751137f admin admin] PATH_INFO: `/v3/projects/569e75116c1645dbb9330edb91ffe91f/users/a06c65bf5b7845199a98d056650f7a3d/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:53.305539 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d3f57-6d19-4c0f-9080-ced4f751137f admin admin] RBAC: Authorizing `identity:create_grant(project_id=569e75116c1645dbb9330edb91ffe91f, user_id=a06c65bf5b7845199a98d056650f7a3d, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:53.308352 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d3f57-6d19-4c0f-9080-ced4f751137f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:53.322255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1037/2080] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:41:53 2022] PUT /identity/v3/projects/569e75116c1645dbb9330edb91ffe91f/users/a06c65bf5b7845199a98d056650f7a3d/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:41:53.333114 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bbda8-6c87-4970-a777-4393d22396d5 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:53.333437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bbda8-6c87-4970-a777-4393d22396d5 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:53.333741 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bbda8-6c87-4970-a777-4393d22396d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:53.361898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-0e5bbda8-6c87-4970-a777-4393d22396d5 None None] MFA Rules not processed for user `a06c65bf5b7845199a98d056650f7a3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:41:53.363114 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-0e5bbda8-6c87-4970-a777-4393d22396d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:53.436773 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1044/2081] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:41:53 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:41:53.441321 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d3f57-6d19-4c0f-9080-ced4f751137f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:53.555393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d3f57-6d19-4c0f-9080-ced4f751137f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:53.561930 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91153c-7609-476a-ac92-d07a6ad7e2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:53.564314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91153c-7609-476a-ac92-d07a6ad7e2a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:53.565193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91153c-7609-476a-ac92-d07a6ad7e2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:53.569764 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91153c-7609-476a-ac92-d07a6ad7e2a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:53.570483 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91153c-7609-476a-ac92-d07a6ad7e2a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:53.572477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91153c-7609-476a-ac92-d07a6ad7e2a9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:53.608719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1038/2082] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:41:53 2022] POST /identity/v3/projects => generated 389 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:41:53.609956 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bbda8-6c87-4970-a777-4393d22396d5 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:53.634038 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bbda8-6c87-4970-a777-4393d22396d5 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:53.686998 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:41:53.745685 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc83f39-f708-4d78-a4dd-38c6c6b179ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:53.746417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc83f39-f708-4d78-a4dd-38c6c6b179ba admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:53.746796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc83f39-f708-4d78-a4dd-38c6c6b179ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:53.747165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc83f39-f708-4d78-a4dd-38c6c6b179ba admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:53.747916 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc83f39-f708-4d78-a4dd-38c6c6b179ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:53.750037 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc83f39-f708-4d78-a4dd-38c6c6b179ba admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:53.776935 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1045/2083] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:41:53 2022] POST /identity/v3/users => generated 301 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:41:53.780970 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91153c-7609-476a-ac92-d07a6ad7e2a9 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:53.803079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91153c-7609-476a-ac92-d07a6ad7e2a9 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:53.807809 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e6c76-d9c5-457e-b47f-2ed02a0b837c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:53.808420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e6c76-d9c5-457e-b47f-2ed02a0b837c admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:53.808701 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e6c76-d9c5-457e-b47f-2ed02a0b837c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:53.809005 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e6c76-d9c5-457e-b47f-2ed02a0b837c admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:53.809654 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e6c76-d9c5-457e-b47f-2ed02a0b837c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:53.811262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e6c76-d9c5-457e-b47f-2ed02a0b837c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:53.840888 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1039/2084] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:41:53 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:41:53.842046 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc83f39-f708-4d78-a4dd-38c6c6b179ba admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:53.853969 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc83f39-f708-4d78-a4dd-38c6c6b179ba admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:53.859844 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db717b-cc2f-4cf8-9997-0cf95579ecf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:53.861204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db717b-cc2f-4cf8-9997-0cf95579ecf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:53.861679 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db717b-cc2f-4cf8-9997-0cf95579ecf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:53.862124 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db717b-cc2f-4cf8-9997-0cf95579ecf2 admin admin] PATH_INFO: `/v3/projects/b667ded9078842e88bec05e156c77df9/users/1ac785ce88ef4b78b3f5fe402a22d26f/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:53.884108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db717b-cc2f-4cf8-9997-0cf95579ecf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b667ded9078842e88bec05e156c77df9, user_id=1ac785ce88ef4b78b3f5fe402a22d26f, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:53.887182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db717b-cc2f-4cf8-9997-0cf95579ecf2 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:53.906695 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e6c76-d9c5-457e-b47f-2ed02a0b837c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:53.913424 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1046/2085] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:41:53 2022] PUT /identity/v3/projects/b667ded9078842e88bec05e156c77df9/users/1ac785ce88ef4b78b3f5fe402a22d26f/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:41:53.918791 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e6c76-d9c5-457e-b47f-2ed02a0b837c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:53.934917 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:41:53.992217 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ee958-621f-4007-b39b-bacc8fc7781d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:53.993183 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ee958-621f-4007-b39b-bacc8fc7781d admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:53.993657 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ee958-621f-4007-b39b-bacc8fc7781d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:53.994282 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ee958-621f-4007-b39b-bacc8fc7781d admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:53.995393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ee958-621f-4007-b39b-bacc8fc7781d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:53.997274 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ee958-621f-4007-b39b-bacc8fc7781d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:54.010615 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1040/2086] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:41:53 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:41:54.014637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db717b-cc2f-4cf8-9997-0cf95579ecf2 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:54.030593 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db717b-cc2f-4cf8-9997-0cf95579ecf2 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:54.037478 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653455f2-238d-4fa7-99ef-750e29693102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:54.038157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653455f2-238d-4fa7-99ef-750e29693102 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:54.038486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653455f2-238d-4fa7-99ef-750e29693102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:54.038772 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653455f2-238d-4fa7-99ef-750e29693102 admin admin] PATH_INFO: `/v3/projects/b667ded9078842e88bec05e156c77df9/users/1ac785ce88ef4b78b3f5fe402a22d26f/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:54.040754 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653455f2-238d-4fa7-99ef-750e29693102 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b667ded9078842e88bec05e156c77df9, user_id=1ac785ce88ef4b78b3f5fe402a22d26f, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:54.043958 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653455f2-238d-4fa7-99ef-750e29693102 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:54.067223 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1047/2087] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:41:54 2022] PUT /identity/v3/projects/b667ded9078842e88bec05e156c77df9/users/1ac785ce88ef4b78b3f5fe402a22d26f/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:41:54.073106 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fe279-2a92-4d8f-a8d6-75bdf5e6efa9 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:54.073446 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fe279-2a92-4d8f-a8d6-75bdf5e6efa9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:54.073743 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fe279-2a92-4d8f-a8d6-75bdf5e6efa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:54.117996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a06fe279-2a92-4d8f-a8d6-75bdf5e6efa9 None None] MFA Rules not processed for user `1ac785ce88ef4b78b3f5fe402a22d26f`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:41:54.137620 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a06fe279-2a92-4d8f-a8d6-75bdf5e6efa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:54.242263 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1041/2088] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:41:54 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:41:54.253619 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653455f2-238d-4fa7-99ef-750e29693102 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:54.255787 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-653455f2-238d-4fa7-99ef-750e29693102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:54.323976 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653455f2-238d-4fa7-99ef-750e29693102 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:54.327785 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ac36a3-c85b-4294-8f99-f8da0277a697 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:54.328427 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac36a3-c85b-4294-8f99-f8da0277a697 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:54.328682 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac36a3-c85b-4294-8f99-f8da0277a697 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:54.328991 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ac36a3-c85b-4294-8f99-f8da0277a697 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:54.339276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac36a3-c85b-4294-8f99-f8da0277a697 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:54.340740 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ac36a3-c85b-4294-8f99-f8da0277a697 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:54.351899 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1048/2089] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:41:54 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:41:54.506627 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fe279-2a92-4d8f-a8d6-75bdf5e6efa9 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:54.574862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fe279-2a92-4d8f-a8d6-75bdf5e6efa9 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:54.580609 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4d5fae-a99c-4e74-a603-eaba3a10b2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:54.581399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d5fae-a99c-4e74-a603-eaba3a10b2fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:54.581720 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d5fae-a99c-4e74-a603-eaba3a10b2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:54.581936 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d5fae-a99c-4e74-a603-eaba3a10b2fb admin admin] PATH_INFO: `/v3/users/a06c65bf5b7845199a98d056650f7a3d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:54.583631 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4d5fae-a99c-4e74-a603-eaba3a10b2fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a06c65bf5b7845199a98d056650f7a3d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:54.585685 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4d5fae-a99c-4e74-a603-eaba3a10b2fb admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:54.645839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:41:54.654734 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cc4d5fae-a99c-4e74-a603-eaba3a10b2fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a06c65bf5b7845199a98d056650f7a3d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:54.655463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cc4d5fae-a99c-4e74-a603-eaba3a10b2fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a06c65bf5b7845199a98d056650f7a3d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:54.663690 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cc4d5fae-a99c-4e74-a603-eaba3a10b2fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a06c65bf5b7845199a98d056650f7a3d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:54.677949 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cc4d5fae-a99c-4e74-a603-eaba3a10b2fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a06c65bf5b7845199a98d056650f7a3d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:54.678826 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-cc4d5fae-a99c-4e74-a603-eaba3a10b2fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a06c65bf5b7845199a98d056650f7a3d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:54.717657 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1042/2090] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:41:54 2022] DELETE /identity/v3/users/a06c65bf5b7845199a98d056650f7a3d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:41:54.719225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d1c9b-6696-4ba7-bfdd-34db1793a831 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:54.719378 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d1c9b-6696-4ba7-bfdd-34db1793a831 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:54.719657 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d1c9b-6696-4ba7-bfdd-34db1793a831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:54.749244 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-4c4d1c9b-6696-4ba7-bfdd-34db1793a831 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:41:54.752980 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-4c4d1c9b-6696-4ba7-bfdd-34db1793a831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:54.858218 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1049/2091] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:41:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:41:54.866203 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4d5fae-a99c-4e74-a603-eaba3a10b2fb admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:54.867725 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cc4d5fae-a99c-4e74-a603-eaba3a10b2fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:54.947135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4d5fae-a99c-4e74-a603-eaba3a10b2fb admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:54.950778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7c288-171c-411e-8248-4715f31789f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:54.951385 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7c288-171c-411e-8248-4715f31789f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:54.951659 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7c288-171c-411e-8248-4715f31789f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:54.951930 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7c288-171c-411e-8248-4715f31789f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:54.962253 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7c288-171c-411e-8248-4715f31789f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:54.963695 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7c288-171c-411e-8248-4715f31789f8 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:54.974752 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1043/2092] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:41:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:41:55.343093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d1c9b-6696-4ba7-bfdd-34db1793a831 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:55.344896 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-4c4d1c9b-6696-4ba7-bfdd-34db1793a831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:55.358341 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d1c9b-6696-4ba7-bfdd-34db1793a831 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:55.363880 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64a2cc-9da0-4fae-872d-7cae49e0b64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:55.364543 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64a2cc-9da0-4fae-872d-7cae49e0b64b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:55.364823 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64a2cc-9da0-4fae-872d-7cae49e0b64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:55.365125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64a2cc-9da0-4fae-872d-7cae49e0b64b admin admin] PATH_INFO: `/v3/projects/569e75116c1645dbb9330edb91ffe91f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:55.366290 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64a2cc-9da0-4fae-872d-7cae49e0b64b admin admin] RBAC: Authorizing `identity:delete_project(project_id=569e75116c1645dbb9330edb91ffe91f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:55.368394 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64a2cc-9da0-4fae-872d-7cae49e0b64b admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:55.421359 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:41:55.460845 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-7d64a2cc-9da0-4fae-872d-7cae49e0b64b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '569e75116c1645dbb9330edb91ffe91f', 'request_id': 'req-7d64a2cc-9da0-4fae-872d-7cae49e0b64b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:55.470332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-7d64a2cc-9da0-4fae-872d-7cae49e0b64b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '569e75116c1645dbb9330edb91ffe91f', 'request_id': 'req-7d64a2cc-9da0-4fae-872d-7cae49e0b64b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:55.491357 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-7d64a2cc-9da0-4fae-872d-7cae49e0b64b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '569e75116c1645dbb9330edb91ffe91f', 'request_id': 'req-7d64a2cc-9da0-4fae-872d-7cae49e0b64b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:55.513479 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-7d64a2cc-9da0-4fae-872d-7cae49e0b64b admin admin] The token cache is being invalidate because project 569e75116c1645dbb9330edb91ffe91f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:41:55.514005 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-7d64a2cc-9da0-4fae-872d-7cae49e0b64b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:55.514817 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-7d64a2cc-9da0-4fae-872d-7cae49e0b64b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:55.520240 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1050/2093] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:41:55 2022] DELETE /identity/v3/projects/569e75116c1645dbb9330edb91ffe91f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:41:55.521571 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7c288-171c-411e-8248-4715f31789f8 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:55.533062 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7c288-171c-411e-8248-4715f31789f8 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:55.615898 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfc65b-c007-4daf-97c4-f5e649a3c340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:55.617509 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfc65b-c007-4daf-97c4-f5e649a3c340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:55.617787 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfc65b-c007-4daf-97c4-f5e649a3c340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:55.618063 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfc65b-c007-4daf-97c4-f5e649a3c340 admin admin] PATH_INFO: `/v3/users/1ac785ce88ef4b78b3f5fe402a22d26f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:55.619534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dfc65b-c007-4daf-97c4-f5e649a3c340 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ac785ce88ef4b78b3f5fe402a22d26f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:55.623919 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dfc65b-c007-4daf-97c4-f5e649a3c340 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:55.685060 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-58dfc65b-c007-4daf-97c4-f5e649a3c340 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ac785ce88ef4b78b3f5fe402a22d26f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:55.685777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-58dfc65b-c007-4daf-97c4-f5e649a3c340 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ac785ce88ef4b78b3f5fe402a22d26f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:55.693696 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-58dfc65b-c007-4daf-97c4-f5e649a3c340 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ac785ce88ef4b78b3f5fe402a22d26f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:55.701456 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-58dfc65b-c007-4daf-97c4-f5e649a3c340 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ac785ce88ef4b78b3f5fe402a22d26f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:55.702078 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-58dfc65b-c007-4daf-97c4-f5e649a3c340 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ac785ce88ef4b78b3f5fe402a22d26f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:55.703432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:41:55.725984 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1044/2094] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:41:55 2022] DELETE /identity/v3/users/1ac785ce88ef4b78b3f5fe402a22d26f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:41:56.273494 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64a2cc-9da0-4fae-872d-7cae49e0b64b admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:56.275086 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-7d64a2cc-9da0-4fae-872d-7cae49e0b64b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:56.296642 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64a2cc-9da0-4fae-872d-7cae49e0b64b admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:56.365047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917adbe5-3c9a-483b-b952-0894f88aed60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:56.365852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917adbe5-3c9a-483b-b952-0894f88aed60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:56.366202 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917adbe5-3c9a-483b-b952-0894f88aed60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:56.366545 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917adbe5-3c9a-483b-b952-0894f88aed60 admin admin] PATH_INFO: `/v3/projects/b667ded9078842e88bec05e156c77df9` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:56.367856 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917adbe5-3c9a-483b-b952-0894f88aed60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b667ded9078842e88bec05e156c77df9)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:56.370168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917adbe5-3c9a-483b-b952-0894f88aed60 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:56.436229 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-917adbe5-3c9a-483b-b952-0894f88aed60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b667ded9078842e88bec05e156c77df9', 'request_id': 'req-917adbe5-3c9a-483b-b952-0894f88aed60'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:56.444524 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-917adbe5-3c9a-483b-b952-0894f88aed60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b667ded9078842e88bec05e156c77df9', 'request_id': 'req-917adbe5-3c9a-483b-b952-0894f88aed60'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:56.459763 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-917adbe5-3c9a-483b-b952-0894f88aed60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b667ded9078842e88bec05e156c77df9', 'request_id': 'req-917adbe5-3c9a-483b-b952-0894f88aed60'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:56.479464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-917adbe5-3c9a-483b-b952-0894f88aed60 admin admin] The token cache is being invalidate because project b667ded9078842e88bec05e156c77df9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:41:56.479813 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-917adbe5-3c9a-483b-b952-0894f88aed60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:56.480421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-917adbe5-3c9a-483b-b952-0894f88aed60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:41:56.482178 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1051/2095] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:41:56 2022] DELETE /identity/v3/projects/b667ded9078842e88bec05e156c77df9 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:41:56.487112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84255d-afe1-4073-9849-639922713ae2 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:56.487562 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84255d-afe1-4073-9849-639922713ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:56.487929 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84255d-afe1-4073-9849-639922713ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:56.518281 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-6b84255d-afe1-4073-9849-639922713ae2 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:41:56.519614 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6b84255d-afe1-4073-9849-639922713ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:56.593521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1045/2096] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:41:56.603267 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc3105-deda-4606-a0bc-8a32d2d6a63b None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:56.603643 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc3105-deda-4606-a0bc-8a32d2d6a63b None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:56.603925 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc3105-deda-4606-a0bc-8a32d2d6a63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:56.637999 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-95fc3105-deda-4606-a0bc-8a32d2d6a63b None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:41:56.663160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-95fc3105-deda-4606-a0bc-8a32d2d6a63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:56.692485 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1052/2097] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:41:56.694398 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84255d-afe1-4073-9849-639922713ae2 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:56.713274 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84255d-afe1-4073-9849-639922713ae2 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:56.719764 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499023fd-3bab-4ba1-af94-5a6cd2a7f758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:56.720458 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499023fd-3bab-4ba1-af94-5a6cd2a7f758 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:56.720842 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499023fd-3bab-4ba1-af94-5a6cd2a7f758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:56.721166 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499023fd-3bab-4ba1-af94-5a6cd2a7f758 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:56.721927 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499023fd-3bab-4ba1-af94-5a6cd2a7f758 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:41:56.722234 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499023fd-3bab-4ba1-af94-5a6cd2a7f758 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:56.727167 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499023fd-3bab-4ba1-af94-5a6cd2a7f758 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:56.759957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1046/2098] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:41:56.761526 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc3105-deda-4606-a0bc-8a32d2d6a63b None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:56.771978 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc3105-deda-4606-a0bc-8a32d2d6a63b None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:56.776723 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ef2ce-f3da-4058-9c3f-1b1ab2be8aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:56.777363 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ef2ce-f3da-4058-9c3f-1b1ab2be8aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:56.777643 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ef2ce-f3da-4058-9c3f-1b1ab2be8aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:56.777920 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ef2ce-f3da-4058-9c3f-1b1ab2be8aa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:56.778556 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40ef2ce-f3da-4058-9c3f-1b1ab2be8aa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:56.780466 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40ef2ce-f3da-4058-9c3f-1b1ab2be8aa5 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:56.813860 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1053/2099] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:41:56 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:41:56.815391 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499023fd-3bab-4ba1-af94-5a6cd2a7f758 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:56.828271 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499023fd-3bab-4ba1-af94-5a6cd2a7f758 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:56.899442 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb8568-8b73-409d-b3fa-71b5384876c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:56.899966 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb8568-8b73-409d-b3fa-71b5384876c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:56.900244 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb8568-8b73-409d-b3fa-71b5384876c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:56.900517 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb8568-8b73-409d-b3fa-71b5384876c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:56.901189 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb8568-8b73-409d-b3fa-71b5384876c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:56.903105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb8568-8b73-409d-b3fa-71b5384876c0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:56.927591 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1047/2100] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:41:56 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:41:56.932226 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ef2ce-f3da-4058-9c3f-1b1ab2be8aa5 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:56.941901 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:41:56.947511 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ef2ce-f3da-4058-9c3f-1b1ab2be8aa5 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:56.956263 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af2997-9f1d-4bee-a097-14b477124dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:56.958231 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af2997-9f1d-4bee-a097-14b477124dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:56.958798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af2997-9f1d-4bee-a097-14b477124dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:56.959486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af2997-9f1d-4bee-a097-14b477124dd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:56.960265 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af2997-9f1d-4bee-a097-14b477124dd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:56.962129 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af2997-9f1d-4bee-a097-14b477124dd3 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:56.987905 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1054/2101] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:41:56.992422 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb8568-8b73-409d-b3fa-71b5384876c0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:57.002944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb8568-8b73-409d-b3fa-71b5384876c0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:57.007743 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4335d-0902-45dc-9a35-300d72ae9e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:57.008347 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4335d-0902-45dc-9a35-300d72ae9e74 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:57.008629 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4335d-0902-45dc-9a35-300d72ae9e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:57.008933 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4335d-0902-45dc-9a35-300d72ae9e74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:57.009612 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4335d-0902-45dc-9a35-300d72ae9e74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:57.011226 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4335d-0902-45dc-9a35-300d72ae9e74 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:57.024904 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1048/2102] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:41:57.028759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af2997-9f1d-4bee-a097-14b477124dd3 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:57.040565 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af2997-9f1d-4bee-a097-14b477124dd3 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:57.045822 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d49b28-cee4-4895-b8b1-3504528d02b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:57.046553 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d49b28-cee4-4895-b8b1-3504528d02b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:57.046838 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d49b28-cee4-4895-b8b1-3504528d02b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:57.047111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d49b28-cee4-4895-b8b1-3504528d02b7 admin admin] PATH_INFO: `/v3/projects/a444b7fa425b4f0ba7833b93607daa25/users/69c04718cd82496abbc82e353873e604/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:57.063180 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d49b28-cee4-4895-b8b1-3504528d02b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a444b7fa425b4f0ba7833b93607daa25, user_id=69c04718cd82496abbc82e353873e604, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:57.065952 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d49b28-cee4-4895-b8b1-3504528d02b7 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:57.082605 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1055/2103] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:41:57 2022] PUT /identity/v3/projects/a444b7fa425b4f0ba7833b93607daa25/users/69c04718cd82496abbc82e353873e604/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:41:57.087479 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e1f9d-e86b-474b-bfdf-2ef53ea31e20 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:57.087894 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e1f9d-e86b-474b-bfdf-2ef53ea31e20 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:57.088222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e1f9d-e86b-474b-bfdf-2ef53ea31e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:57.110951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-fb7e1f9d-e86b-474b-bfdf-2ef53ea31e20 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:41:57.112168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fb7e1f9d-e86b-474b-bfdf-2ef53ea31e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:57.174908 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:41:57.200452 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1049/2104] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:41:57.202940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d49b28-cee4-4895-b8b1-3504528d02b7 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:57.203973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-61d49b28-cee4-4895-b8b1-3504528d02b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:57.273397 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d49b28-cee4-4895-b8b1-3504528d02b7 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:57.277771 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f20f8a-13d6-40e8-a000-f8e6306d22f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:57.278387 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f20f8a-13d6-40e8-a000-f8e6306d22f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:57.278680 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f20f8a-13d6-40e8-a000-f8e6306d22f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:57.278979 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f20f8a-13d6-40e8-a000-f8e6306d22f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:57.290408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f20f8a-13d6-40e8-a000-f8e6306d22f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:57.292711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f20f8a-13d6-40e8-a000-f8e6306d22f7 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:57.305973 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1056/2105] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:41:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:41:57.981036 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a8469-fc40-46fc-a436-2b48bde86b0d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:57.981750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a8469-fc40-46fc-a436-2b48bde86b0d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:57.981750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a8469-fc40-46fc-a436-2b48bde86b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:58.017603 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-639a8469-fc40-46fc-a436-2b48bde86b0d None None] MFA Rules not processed for user `69c04718cd82496abbc82e353873e604`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:41:58.019338 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-639a8469-fc40-46fc-a436-2b48bde86b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:58.098242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1050/2106] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:41:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:41:58.109510 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f20f8a-13d6-40e8-a000-f8e6306d22f7 service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:41:58.111044 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-95f20f8a-13d6-40e8-a000-f8e6306d22f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:58.183285 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f20f8a-13d6-40e8-a000-f8e6306d22f7 service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:41:58.190761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba39654-ff6d-4056-aa09-deaf3ed7cf88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:41:58.191755 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba39654-ff6d-4056-aa09-deaf3ed7cf88 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:58.192213 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba39654-ff6d-4056-aa09-deaf3ed7cf88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:58.192699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba39654-ff6d-4056-aa09-deaf3ed7cf88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:58.205276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba39654-ff6d-4056-aa09-deaf3ed7cf88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:41:58.206811 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba39654-ff6d-4056-aa09-deaf3ed7cf88 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:41:58.222029 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1057/2107] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:41:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:41:59.706203 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8e845-3e02-4bb3-879e-c15ebcabd33c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:41:59.706782 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8e845-3e02-4bb3-879e-c15ebcabd33c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:41:59.706782 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8e845-3e02-4bb3-879e-c15ebcabd33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:41:59.731251 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9ad8e845-3e02-4bb3-879e-c15ebcabd33c None None] MFA Rules not processed for user `69c04718cd82496abbc82e353873e604`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:41:59.732795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9ad8e845-3e02-4bb3-879e-c15ebcabd33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:41:59.741640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1051/2108] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:41:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:04.199353 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba39654-ff6d-4056-aa09-deaf3ed7cf88 service cinder] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:04.210132 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba39654-ff6d-4056-aa09-deaf3ed7cf88 service cinder] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:04.213771 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef0544-cf3b-49ba-999f-545c6441d263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:04.214373 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef0544-cf3b-49ba-999f-545c6441d263 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:04.214649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef0544-cf3b-49ba-999f-545c6441d263 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:04.214920 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef0544-cf3b-49ba-999f-545c6441d263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:04.216644 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-adef0544-cf3b-49ba-999f-545c6441d263 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:04.230398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef0544-cf3b-49ba-999f-545c6441d263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:04.231874 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef0544-cf3b-49ba-999f-545c6441d263 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:04.242622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1058/2109] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:42:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:42:07.137596 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a871dc-51ab-492e-a241-bfd9300a3d3a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:07.138097 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a871dc-51ab-492e-a241-bfd9300a3d3a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:07.138183 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a871dc-51ab-492e-a241-bfd9300a3d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:07.159844 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-51a871dc-51ab-492e-a241-bfd9300a3d3a None None] MFA Rules not processed for user `85bb137d0fb74031a49a5c629b32c32b`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:07.161310 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-51a871dc-51ab-492e-a241-bfd9300a3d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:07.220610 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1052/2110] 10.222.0.23 () {58 vars in 967 bytes} [Thu Oct 20 13:42:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:07.234671 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef0544-cf3b-49ba-999f-545c6441d263 service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:07.236056 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-adef0544-cf3b-49ba-999f-545c6441d263 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:07.324717 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:07.330408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef0544-cf3b-49ba-999f-545c6441d263 service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:07.339917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dedd82-cfed-40c7-b407-fa387d421f7d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9834211488354651803000c7e8fab80b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:07.340811 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dedd82-cfed-40c7-b407-fa387d421f7d service swift] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:07.341533 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dedd82-cfed-40c7-b407-fa387d421f7d service swift] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:07.341926 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dedd82-cfed-40c7-b407-fa387d421f7d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:07.353241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dedd82-cfed-40c7-b407-fa387d421f7d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:07.355504 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dedd82-cfed-40c7-b407-fa387d421f7d service swift] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:07.368106 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1059/2111] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 20 13:42:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 13:42:09.116669 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a871dc-51ab-492e-a241-bfd9300a3d3a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:09.122445 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-51a871dc-51ab-492e-a241-bfd9300a3d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:09.127272 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dedd82-cfed-40c7-b407-fa387d421f7d service swift] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:09.130357 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b0dedd82-cfed-40c7-b407-fa387d421f7d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:09.164430 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dedd82-cfed-40c7-b407-fa387d421f7d service swift] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:09.183430 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9412d65-05fc-45ee-aee4-7cbebc726799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:09.184086 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9412d65-05fc-45ee-aee4-7cbebc726799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:09.184398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9412d65-05fc-45ee-aee4-7cbebc726799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:09.184673 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9412d65-05fc-45ee-aee4-7cbebc726799 admin admin] PATH_INFO: `/v3/users/fe56d1d4278c43cd96a39f7f75f373b8` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:09.193905 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9412d65-05fc-45ee-aee4-7cbebc726799 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe56d1d4278c43cd96a39f7f75f373b8)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:09.199204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9412d65-05fc-45ee-aee4-7cbebc726799 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:09.308346 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-f9412d65-05fc-45ee-aee4-7cbebc726799 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe56d1d4278c43cd96a39f7f75f373b8'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:09.309269 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-f9412d65-05fc-45ee-aee4-7cbebc726799 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe56d1d4278c43cd96a39f7f75f373b8'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:09.320087 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-f9412d65-05fc-45ee-aee4-7cbebc726799 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe56d1d4278c43cd96a39f7f75f373b8'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:09.320712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-f9412d65-05fc-45ee-aee4-7cbebc726799 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe56d1d4278c43cd96a39f7f75f373b8'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:09.330203 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a871dc-51ab-492e-a241-bfd9300a3d3a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:09.337642 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-f9412d65-05fc-45ee-aee4-7cbebc726799 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe56d1d4278c43cd96a39f7f75f373b8'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:09.347967 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f3852-38b6-4bc0-ae67-4f24dd93a105 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2440594934b64621a01c441133778155', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:09.348602 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f3852-38b6-4bc0-ae67-4f24dd93a105 service placement] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:09.348903 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f3852-38b6-4bc0-ae67-4f24dd93a105 service placement] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:09.349187 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f3852-38b6-4bc0-ae67-4f24dd93a105 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:09.351044 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8a4f3852-38b6-4bc0-ae67-4f24dd93a105 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:09.370561 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1060/2112] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:42:09 2022] DELETE /identity/v3/users/fe56d1d4278c43cd96a39f7f75f373b8 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:09.392597 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f3852-38b6-4bc0-ae67-4f24dd93a105 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:09.394299 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f3852-38b6-4bc0-ae67-4f24dd93a105 service placement] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:09.423610 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1053/2113] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:42:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:42:09.699896 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9412d65-05fc-45ee-aee4-7cbebc726799 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:09.701524 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-f9412d65-05fc-45ee-aee4-7cbebc726799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:09.882060 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9412d65-05fc-45ee-aee4-7cbebc726799 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:09.891239 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f64a2a-fe01-4dbd-889a-a1e449883a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:09.892296 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f64a2a-fe01-4dbd-889a-a1e449883a1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:09.892680 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f64a2a-fe01-4dbd-889a-a1e449883a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:09.893044 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f64a2a-fe01-4dbd-889a-a1e449883a1e admin admin] PATH_INFO: `/v3/projects/747a468811fb478aa6c49b848ccee6d0` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:09.903417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f64a2a-fe01-4dbd-889a-a1e449883a1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=747a468811fb478aa6c49b848ccee6d0)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:09.911948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f64a2a-fe01-4dbd-889a-a1e449883a1e admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:10.029280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c3f64a2a-fe01-4dbd-889a-a1e449883a1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '747a468811fb478aa6c49b848ccee6d0', 'request_id': 'req-c3f64a2a-fe01-4dbd-889a-a1e449883a1e'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:10.068447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c3f64a2a-fe01-4dbd-889a-a1e449883a1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '747a468811fb478aa6c49b848ccee6d0', 'request_id': 'req-c3f64a2a-fe01-4dbd-889a-a1e449883a1e'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:10.099381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c3f64a2a-fe01-4dbd-889a-a1e449883a1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '747a468811fb478aa6c49b848ccee6d0', 'request_id': 'req-c3f64a2a-fe01-4dbd-889a-a1e449883a1e'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:10.110377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:10.130849 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c3f64a2a-fe01-4dbd-889a-a1e449883a1e admin admin] The token cache is being invalidate because project 747a468811fb478aa6c49b848ccee6d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:42:10.134724 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c3f64a2a-fe01-4dbd-889a-a1e449883a1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:10.137714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c3f64a2a-fe01-4dbd-889a-a1e449883a1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:10.144752 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1061/2114] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:42:09 2022] DELETE /identity/v3/projects/747a468811fb478aa6c49b848ccee6d0 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:10.146205 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4f397-5b73-4388-b128-7aeebde353a8 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:10.146685 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4f397-5b73-4388-b128-7aeebde353a8 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:10.146811 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4f397-5b73-4388-b128-7aeebde353a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:10.170424 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:10.187875 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-54f4f397-5b73-4388-b128-7aeebde353a8 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:10.189535 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-54f4f397-5b73-4388-b128-7aeebde353a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:10.291119 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1054/2115] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:10.300340 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97a155-2393-4d48-afe0-44120c264575 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:10.300756 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97a155-2393-4d48-afe0-44120c264575 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:10.300943 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97a155-2393-4d48-afe0-44120c264575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:10.323397 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-0c97a155-2393-4d48-afe0-44120c264575 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:10.324640 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-0c97a155-2393-4d48-afe0-44120c264575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:10.343599 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4f397-5b73-4388-b128-7aeebde353a8 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:10.343722 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1062/2116] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:10.350332 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4f397-5b73-4388-b128-7aeebde353a8 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:10.355350 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b537e90-74ee-4c16-be78-865bbd737076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:10.356117 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b537e90-74ee-4c16-be78-865bbd737076 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:10.356467 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b537e90-74ee-4c16-be78-865bbd737076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:10.357073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b537e90-74ee-4c16-be78-865bbd737076 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:10.357847 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b537e90-74ee-4c16-be78-865bbd737076 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:42:10.358226 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b537e90-74ee-4c16-be78-865bbd737076 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:10.360338 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b537e90-74ee-4c16-be78-865bbd737076 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:10.375205 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1055/2117] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:42:10.381929 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97a155-2393-4d48-afe0-44120c264575 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:10.391708 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97a155-2393-4d48-afe0-44120c264575 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:10.397305 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186085e8-cb1e-46a4-9a8e-02ef32fff695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:10.399242 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186085e8-cb1e-46a4-9a8e-02ef32fff695 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:10.399540 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186085e8-cb1e-46a4-9a8e-02ef32fff695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:10.400233 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186085e8-cb1e-46a4-9a8e-02ef32fff695 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:10.400486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186085e8-cb1e-46a4-9a8e-02ef32fff695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:10.402463 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186085e8-cb1e-46a4-9a8e-02ef32fff695 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:10.447947 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1063/2118] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:42:10 2022] POST /identity/v3/projects => generated 403 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:42:10.453835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b537e90-74ee-4c16-be78-865bbd737076 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:10.473441 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b537e90-74ee-4c16-be78-865bbd737076 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:10.548186 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b1577-645f-432a-b5f2-9306319c7882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:10.548816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b1577-645f-432a-b5f2-9306319c7882 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:10.549131 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b1577-645f-432a-b5f2-9306319c7882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:10.549560 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b1577-645f-432a-b5f2-9306319c7882 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:10.550872 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b1577-645f-432a-b5f2-9306319c7882 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:10.553159 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b1577-645f-432a-b5f2-9306319c7882 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:10.579582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1056/2119] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:42:10 2022] POST /identity/v3/users => generated 302 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:42:10.584005 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186085e8-cb1e-46a4-9a8e-02ef32fff695 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:10.610023 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186085e8-cb1e-46a4-9a8e-02ef32fff695 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:10.616559 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d81ee-6e98-43ea-aa40-77b081aadcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:10.617213 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d81ee-6e98-43ea-aa40-77b081aadcaa admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:10.617494 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d81ee-6e98-43ea-aa40-77b081aadcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:10.618018 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d81ee-6e98-43ea-aa40-77b081aadcaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:10.618683 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d81ee-6e98-43ea-aa40-77b081aadcaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:10.620639 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74d81ee-6e98-43ea-aa40-77b081aadcaa admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:10.631851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1064/2120] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:10.635838 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b1577-645f-432a-b5f2-9306319c7882 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:10.647184 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b1577-645f-432a-b5f2-9306319c7882 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:10.654367 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7666e55-af02-495a-b4c5-1ec75700828d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:10.655062 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7666e55-af02-495a-b4c5-1ec75700828d admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:10.655310 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7666e55-af02-495a-b4c5-1ec75700828d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:10.655810 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7666e55-af02-495a-b4c5-1ec75700828d admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:10.656703 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7666e55-af02-495a-b4c5-1ec75700828d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:10.658640 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7666e55-af02-495a-b4c5-1ec75700828d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:10.672198 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1057/2121] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:10.676536 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d81ee-6e98-43ea-aa40-77b081aadcaa admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:10.686800 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d81ee-6e98-43ea-aa40-77b081aadcaa admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:10.692035 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ebe3a-83da-46f6-b9cc-a3bfb02f685b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:10.692698 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ebe3a-83da-46f6-b9cc-a3bfb02f685b admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:10.693131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ebe3a-83da-46f6-b9cc-a3bfb02f685b admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:10.693408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ebe3a-83da-46f6-b9cc-a3bfb02f685b admin admin] PATH_INFO: `/v3/projects/2038d3ff419446cfb16ec2347bc90950/users/8e180ff6977541468ecaefce6f2f76a4/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:10.717094 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ebe3a-83da-46f6-b9cc-a3bfb02f685b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2038d3ff419446cfb16ec2347bc90950, user_id=8e180ff6977541468ecaefce6f2f76a4, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:10.721247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ebe3a-83da-46f6-b9cc-a3bfb02f685b admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:10.747941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1065/2122] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:10 2022] PUT /identity/v3/projects/2038d3ff419446cfb16ec2347bc90950/users/8e180ff6977541468ecaefce6f2f76a4/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:10.752827 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c938a6-4a47-427d-8001-68afea3b66d3 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:10.753752 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c938a6-4a47-427d-8001-68afea3b66d3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:10.754034 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c938a6-4a47-427d-8001-68afea3b66d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:10.793292 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-46c938a6-4a47-427d-8001-68afea3b66d3 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:10.794541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-46c938a6-4a47-427d-8001-68afea3b66d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:10.894033 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1058/2123] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:10.905941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ebe3a-83da-46f6-b9cc-a3bfb02f685b admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:10.907705 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-501ebe3a-83da-46f6-b9cc-a3bfb02f685b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:11.005291 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ebe3a-83da-46f6-b9cc-a3bfb02f685b admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:11.009583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e936e-d06a-49ae-a889-d233cac03377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:11.010207 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e936e-d06a-49ae-a889-d233cac03377 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:11.010485 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e936e-d06a-49ae-a889-d233cac03377 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:11.010752 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e936e-d06a-49ae-a889-d233cac03377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:11.024186 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e936e-d06a-49ae-a889-d233cac03377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:11.025850 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e936e-d06a-49ae-a889-d233cac03377 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:11.038576 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1066/2124] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:42:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:42:11.438200 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c938a6-4a47-427d-8001-68afea3b66d3 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:11.439679 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-46c938a6-4a47-427d-8001-68afea3b66d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:11.454122 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:11.563841 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c938a6-4a47-427d-8001-68afea3b66d3 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:11.574536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0e832-106d-4cd3-9ce0-125a5fe815d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2440594934b64621a01c441133778155', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:11.575951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0e832-106d-4cd3-9ce0-125a5fe815d3 service placement] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:11.576281 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0e832-106d-4cd3-9ce0-125a5fe815d3 service placement] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:11.576724 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0e832-106d-4cd3-9ce0-125a5fe815d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:11.579352 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-53a0e832-106d-4cd3-9ce0-125a5fe815d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:11.602621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0e832-106d-4cd3-9ce0-125a5fe815d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:11.604688 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0e832-106d-4cd3-9ce0-125a5fe815d3 service placement] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:11.622291 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1059/2125] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:42:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:42:11.986430 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6981ef4-ec55-4764-9e21-d910c506ff00 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:11.986911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6981ef4-ec55-4764-9e21-d910c506ff00 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:11.987003 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6981ef4-ec55-4764-9e21-d910c506ff00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:12.017109 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-b6981ef4-ec55-4764-9e21-d910c506ff00 None None] MFA Rules not processed for user `8e180ff6977541468ecaefce6f2f76a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:12.018341 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b6981ef4-ec55-4764-9e21-d910c506ff00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:12.115460 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1067/2126] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:11 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:12.121342 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0e832-106d-4cd3-9ce0-125a5fe815d3 service placement] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:12.134540 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0e832-106d-4cd3-9ce0-125a5fe815d3 service placement] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:12.140181 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14763f2a-4ac3-4591-81e0-b6442067636d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:12.141471 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14763f2a-4ac3-4591-81e0-b6442067636d admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:12.141948 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14763f2a-4ac3-4591-81e0-b6442067636d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:12.142424 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14763f2a-4ac3-4591-81e0-b6442067636d admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:12.143310 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14763f2a-4ac3-4591-81e0-b6442067636d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:12.146593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14763f2a-4ac3-4591-81e0-b6442067636d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:12.182489 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1060/2127] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:42:12 2022] POST /identity/v3/projects => generated 403 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:42:12.183679 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6981ef4-ec55-4764-9e21-d910c506ff00 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:12.198603 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6981ef4-ec55-4764-9e21-d910c506ff00 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:12.220505 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:12.273761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb910f9b-102c-4225-952d-02dc717e364d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:12.274380 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb910f9b-102c-4225-952d-02dc717e364d admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:12.274657 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb910f9b-102c-4225-952d-02dc717e364d admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:12.274925 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb910f9b-102c-4225-952d-02dc717e364d admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:12.275570 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb910f9b-102c-4225-952d-02dc717e364d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:12.277548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb910f9b-102c-4225-952d-02dc717e364d admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:12.306541 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1068/2128] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:42:12 2022] POST /identity/v3/users => generated 308 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:42:12.307754 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14763f2a-4ac3-4591-81e0-b6442067636d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:12.323086 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14763f2a-4ac3-4591-81e0-b6442067636d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:12.329659 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e2314-6405-4382-9e53-5ca980ea43cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:12.330274 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e2314-6405-4382-9e53-5ca980ea43cc admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:12.330555 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e2314-6405-4382-9e53-5ca980ea43cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:12.330829 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e2314-6405-4382-9e53-5ca980ea43cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:12.332216 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e2314-6405-4382-9e53-5ca980ea43cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:12.334497 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e2314-6405-4382-9e53-5ca980ea43cc admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:12.348992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1061/2129] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:12.350094 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb910f9b-102c-4225-952d-02dc717e364d admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:12.364083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb910f9b-102c-4225-952d-02dc717e364d admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:12.369185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e25248-18e3-44ac-bfb5-80ea47b0a536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:12.369895 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e25248-18e3-44ac-bfb5-80ea47b0a536 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:12.370174 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e25248-18e3-44ac-bfb5-80ea47b0a536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:12.370451 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e25248-18e3-44ac-bfb5-80ea47b0a536 admin admin] PATH_INFO: `/v3/projects/f8ba09eb2ebe434f884edd81e24f8143/users/89dfcdc081c94bc39a40f87883903b51/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:12.392497 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e25248-18e3-44ac-bfb5-80ea47b0a536 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8ba09eb2ebe434f884edd81e24f8143, user_id=89dfcdc081c94bc39a40f87883903b51, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:12.395993 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e25248-18e3-44ac-bfb5-80ea47b0a536 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:12.416728 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1069/2130] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:12 2022] PUT /identity/v3/projects/f8ba09eb2ebe434f884edd81e24f8143/users/89dfcdc081c94bc39a40f87883903b51/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:12.418127 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e2314-6405-4382-9e53-5ca980ea43cc admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:12.428623 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e2314-6405-4382-9e53-5ca980ea43cc admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:12.488004 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422fa441-2b31-4a72-8cbd-c92b206498c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:12.488698 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422fa441-2b31-4a72-8cbd-c92b206498c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:12.488889 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422fa441-2b31-4a72-8cbd-c92b206498c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:12.489173 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422fa441-2b31-4a72-8cbd-c92b206498c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:12.489811 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422fa441-2b31-4a72-8cbd-c92b206498c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:12.491403 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422fa441-2b31-4a72-8cbd-c92b206498c9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:12.508741 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1062/2131] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:12.509817 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e25248-18e3-44ac-bfb5-80ea47b0a536 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:12.521435 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e25248-18e3-44ac-bfb5-80ea47b0a536 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:12.526523 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b3f17-0ae1-4f45-ad0b-cd0291c2088d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:12.526665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b3f17-0ae1-4f45-ad0b-cd0291c2088d admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:12.526880 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b3f17-0ae1-4f45-ad0b-cd0291c2088d admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:12.527151 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b3f17-0ae1-4f45-ad0b-cd0291c2088d admin admin] PATH_INFO: `/v3/projects/f8ba09eb2ebe434f884edd81e24f8143/users/89dfcdc081c94bc39a40f87883903b51/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:12.529198 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b3f17-0ae1-4f45-ad0b-cd0291c2088d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8ba09eb2ebe434f884edd81e24f8143, user_id=89dfcdc081c94bc39a40f87883903b51, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:12.531830 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b3f17-0ae1-4f45-ad0b-cd0291c2088d admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:12.542313 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1070/2132] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:12 2022] PUT /identity/v3/projects/f8ba09eb2ebe434f884edd81e24f8143/users/89dfcdc081c94bc39a40f87883903b51/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:13.209733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13616a2b-5d52-4fed-98b0-c34f3398e985 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:13.210075 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13616a2b-5d52-4fed-98b0-c34f3398e985 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:13.210296 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13616a2b-5d52-4fed-98b0-c34f3398e985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:13.246233 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-13616a2b-5d52-4fed-98b0-c34f3398e985 None None] MFA Rules not processed for user `89dfcdc081c94bc39a40f87883903b51`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:13.247456 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-13616a2b-5d52-4fed-98b0-c34f3398e985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:13.313794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:13.343794 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1063/2133] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:13 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:14.770589 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b3f17-0ae1-4f45-ad0b-cd0291c2088d admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:14.926613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b3f17-0ae1-4f45-ad0b-cd0291c2088d admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:14.932760 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bccb0e-aa1c-419c-bc94-b93522747b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:14.933447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bccb0e-aa1c-419c-bc94-b93522747b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:14.933728 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bccb0e-aa1c-419c-bc94-b93522747b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:14.934121 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bccb0e-aa1c-419c-bc94-b93522747b01 admin admin] PATH_INFO: `/v3/users/8e180ff6977541468ecaefce6f2f76a4` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:14.935690 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bccb0e-aa1c-419c-bc94-b93522747b01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e180ff6977541468ecaefce6f2f76a4)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:14.938601 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bccb0e-aa1c-419c-bc94-b93522747b01 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:15.026588 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-90bccb0e-aa1c-419c-bc94-b93522747b01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e180ff6977541468ecaefce6f2f76a4'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:15.030416 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-90bccb0e-aa1c-419c-bc94-b93522747b01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e180ff6977541468ecaefce6f2f76a4'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:15.045205 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-90bccb0e-aa1c-419c-bc94-b93522747b01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e180ff6977541468ecaefce6f2f76a4'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:15.046726 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-90bccb0e-aa1c-419c-bc94-b93522747b01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e180ff6977541468ecaefce6f2f76a4'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:15.061157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-90bccb0e-aa1c-419c-bc94-b93522747b01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e180ff6977541468ecaefce6f2f76a4'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:15.076484 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=85,keystone:INSERT=6,keystone:DELETE=9 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:15.092683 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1071/2134] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:42:14 2022] DELETE /identity/v3/users/8e180ff6977541468ecaefce6f2f76a4 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:15.545450 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13616a2b-5d52-4fed-98b0-c34f3398e985 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:15.582678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13616a2b-5d52-4fed-98b0-c34f3398e985 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:15.749631 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f536fe-d8e7-49a2-bf93-e413609ea7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:15.769361 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f536fe-d8e7-49a2-bf93-e413609ea7d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:15.769651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f536fe-d8e7-49a2-bf93-e413609ea7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:15.769925 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f536fe-d8e7-49a2-bf93-e413609ea7d2 admin admin] PATH_INFO: `/v3/projects/2038d3ff419446cfb16ec2347bc90950` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:15.772261 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f536fe-d8e7-49a2-bf93-e413609ea7d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2038d3ff419446cfb16ec2347bc90950)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:15.774287 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f536fe-d8e7-49a2-bf93-e413609ea7d2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:15.892288 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f5f536fe-d8e7-49a2-bf93-e413609ea7d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2038d3ff419446cfb16ec2347bc90950', 'request_id': 'req-f5f536fe-d8e7-49a2-bf93-e413609ea7d2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:15.922564 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f5f536fe-d8e7-49a2-bf93-e413609ea7d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2038d3ff419446cfb16ec2347bc90950', 'request_id': 'req-f5f536fe-d8e7-49a2-bf93-e413609ea7d2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:15.942338 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f5f536fe-d8e7-49a2-bf93-e413609ea7d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2038d3ff419446cfb16ec2347bc90950', 'request_id': 'req-f5f536fe-d8e7-49a2-bf93-e413609ea7d2'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:15.960881 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f5f536fe-d8e7-49a2-bf93-e413609ea7d2 admin admin] The token cache is being invalidate because project 2038d3ff419446cfb16ec2347bc90950 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:42:15.961496 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f5f536fe-d8e7-49a2-bf93-e413609ea7d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:15.962483 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f5f536fe-d8e7-49a2-bf93-e413609ea7d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:15.964526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1064/2135] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:42:15 2022] DELETE /identity/v3/projects/2038d3ff419446cfb16ec2347bc90950 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:15.968216 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bccb0e-aa1c-419c-bc94-b93522747b01 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:15.970415 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-90bccb0e-aa1c-419c-bc94-b93522747b01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:15.995515 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bccb0e-aa1c-419c-bc94-b93522747b01 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:16.120832 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50ebeb-8d8a-4bfd-b90d-afab3d7318d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:16.121903 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50ebeb-8d8a-4bfd-b90d-afab3d7318d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:16.122359 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50ebeb-8d8a-4bfd-b90d-afab3d7318d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:16.123061 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50ebeb-8d8a-4bfd-b90d-afab3d7318d3 admin admin] PATH_INFO: `/v3/users/89dfcdc081c94bc39a40f87883903b51` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:16.124539 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b50ebeb-8d8a-4bfd-b90d-afab3d7318d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89dfcdc081c94bc39a40f87883903b51)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:16.126754 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b50ebeb-8d8a-4bfd-b90d-afab3d7318d3 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:16.211436 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-7b50ebeb-8d8a-4bfd-b90d-afab3d7318d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89dfcdc081c94bc39a40f87883903b51'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:16.212312 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-7b50ebeb-8d8a-4bfd-b90d-afab3d7318d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89dfcdc081c94bc39a40f87883903b51'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:16.223974 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-7b50ebeb-8d8a-4bfd-b90d-afab3d7318d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89dfcdc081c94bc39a40f87883903b51'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:16.224851 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-7b50ebeb-8d8a-4bfd-b90d-afab3d7318d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89dfcdc081c94bc39a40f87883903b51'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:16.234409 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-7b50ebeb-8d8a-4bfd-b90d-afab3d7318d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89dfcdc081c94bc39a40f87883903b51'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:16.257453 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1072/2136] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:42:15 2022] DELETE /identity/v3/users/89dfcdc081c94bc39a40f87883903b51 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:16.469679 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f536fe-d8e7-49a2-bf93-e413609ea7d2 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:16.471367 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f5f536fe-d8e7-49a2-bf93-e413609ea7d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:16.490663 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f536fe-d8e7-49a2-bf93-e413609ea7d2 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:16.554151 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2a074-a946-4454-adb1-2d5b7a2f115f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:16.554952 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2a074-a946-4454-adb1-2d5b7a2f115f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:16.555211 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2a074-a946-4454-adb1-2d5b7a2f115f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:16.555485 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2a074-a946-4454-adb1-2d5b7a2f115f admin admin] PATH_INFO: `/v3/projects/f8ba09eb2ebe434f884edd81e24f8143` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:16.556619 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2a074-a946-4454-adb1-2d5b7a2f115f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8ba09eb2ebe434f884edd81e24f8143)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:16.558581 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2a074-a946-4454-adb1-2d5b7a2f115f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:16.588980 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:16.620094 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-81e2a074-a946-4454-adb1-2d5b7a2f115f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8ba09eb2ebe434f884edd81e24f8143', 'request_id': 'req-81e2a074-a946-4454-adb1-2d5b7a2f115f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:16.627306 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-81e2a074-a946-4454-adb1-2d5b7a2f115f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8ba09eb2ebe434f884edd81e24f8143', 'request_id': 'req-81e2a074-a946-4454-adb1-2d5b7a2f115f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:16.635595 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-81e2a074-a946-4454-adb1-2d5b7a2f115f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8ba09eb2ebe434f884edd81e24f8143', 'request_id': 'req-81e2a074-a946-4454-adb1-2d5b7a2f115f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:16.647451 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-81e2a074-a946-4454-adb1-2d5b7a2f115f admin admin] The token cache is being invalidate because project f8ba09eb2ebe434f884edd81e24f8143 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:42:16.647745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-81e2a074-a946-4454-adb1-2d5b7a2f115f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:16.648400 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-81e2a074-a946-4454-adb1-2d5b7a2f115f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:16.650152 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1065/2137] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:42:16 2022] DELETE /identity/v3/projects/f8ba09eb2ebe434f884edd81e24f8143 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:16.655008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1da640-13b7-4900-b5b8-eeb0a32a8d0f None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:16.655302 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1da640-13b7-4900-b5b8-eeb0a32a8d0f None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:16.655582 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1da640-13b7-4900-b5b8-eeb0a32a8d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:16.675747 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-0a1da640-13b7-4900-b5b8-eeb0a32a8d0f None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:16.676895 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-0a1da640-13b7-4900-b5b8-eeb0a32a8d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:16.771400 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1073/2138] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:16.773354 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ddfac-e9ca-46b7-8a59-c88dcd25b82c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:16.773644 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ddfac-e9ca-46b7-8a59-c88dcd25b82c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:16.773922 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ddfac-e9ca-46b7-8a59-c88dcd25b82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:16.797870 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-712ddfac-e9ca-46b7-8a59-c88dcd25b82c None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:16.799285 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-712ddfac-e9ca-46b7-8a59-c88dcd25b82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:16.808696 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1066/2139] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:42:16.813179 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1da640-13b7-4900-b5b8-eeb0a32a8d0f None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:16.824604 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1da640-13b7-4900-b5b8-eeb0a32a8d0f None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:16.834664 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0334b41-1d65-4f29-a720-773513b2498a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:16.835391 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0334b41-1d65-4f29-a720-773513b2498a admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:16.835751 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0334b41-1d65-4f29-a720-773513b2498a admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:16.836108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0334b41-1d65-4f29-a720-773513b2498a admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:16.836895 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0334b41-1d65-4f29-a720-773513b2498a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:42:16.837307 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0334b41-1d65-4f29-a720-773513b2498a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:16.839031 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0334b41-1d65-4f29-a720-773513b2498a admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:16.850895 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:16.857050 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1074/2140] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:42:16.860885 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ddfac-e9ca-46b7-8a59-c88dcd25b82c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:16.874526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ddfac-e9ca-46b7-8a59-c88dcd25b82c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:16.881316 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225c717-03f1-49e6-8ac2-232ceb78c6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:16.881939 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225c717-03f1-49e6-8ac2-232ceb78c6af admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:16.882221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225c717-03f1-49e6-8ac2-232ceb78c6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:16.882495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225c717-03f1-49e6-8ac2-232ceb78c6af admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:16.883135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9225c717-03f1-49e6-8ac2-232ceb78c6af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:16.885277 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9225c717-03f1-49e6-8ac2-232ceb78c6af admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:16.922505 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0334b41-1d65-4f29-a720-773513b2498a admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:16.924998 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1067/2141] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:42:16 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:42:16.934404 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0334b41-1d65-4f29-a720-773513b2498a admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:17.020385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63494a9-0b7b-4513-9687-7cb1646fb8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:17.021079 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63494a9-0b7b-4513-9687-7cb1646fb8d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:17.021537 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63494a9-0b7b-4513-9687-7cb1646fb8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:17.021812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63494a9-0b7b-4513-9687-7cb1646fb8d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:17.022481 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63494a9-0b7b-4513-9687-7cb1646fb8d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:17.024407 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63494a9-0b7b-4513-9687-7cb1646fb8d7 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:17.048540 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1075/2142] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:42:16 2022] POST /identity/v3/users => generated 294 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:42:17.052516 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225c717-03f1-49e6-8ac2-232ceb78c6af admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:17.063467 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225c717-03f1-49e6-8ac2-232ceb78c6af admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:17.068695 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a61f20-f688-422a-86ce-f2975d844552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:17.069320 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a61f20-f688-422a-86ce-f2975d844552 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:17.069600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a61f20-f688-422a-86ce-f2975d844552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:17.069871 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a61f20-f688-422a-86ce-f2975d844552 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:17.070507 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a61f20-f688-422a-86ce-f2975d844552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:17.073910 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a61f20-f688-422a-86ce-f2975d844552 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:17.089093 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1068/2143] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:17.091759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63494a9-0b7b-4513-9687-7cb1646fb8d7 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:17.106753 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63494a9-0b7b-4513-9687-7cb1646fb8d7 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:17.113888 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ad2ee-6849-4df9-8355-283afd8321db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:17.113888 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ad2ee-6849-4df9-8355-283afd8321db admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:17.114202 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ad2ee-6849-4df9-8355-283afd8321db admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:17.114488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ad2ee-6849-4df9-8355-283afd8321db admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:17.115136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ad2ee-6849-4df9-8355-283afd8321db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:17.116912 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ad2ee-6849-4df9-8355-283afd8321db admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:17.137400 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1076/2144] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:17.140196 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a61f20-f688-422a-86ce-f2975d844552 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:17.153709 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a61f20-f688-422a-86ce-f2975d844552 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:17.160284 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c69e203-154c-4df7-b088-3844a18cc34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:17.161148 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69e203-154c-4df7-b088-3844a18cc34e admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:17.161542 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69e203-154c-4df7-b088-3844a18cc34e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:17.161835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69e203-154c-4df7-b088-3844a18cc34e admin admin] PATH_INFO: `/v3/projects/d6131b219fbf469bad78af5f21aaf4fe/users/e2ee317642aa4ddd9b90ce863d5d16e3/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:17.187946 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c69e203-154c-4df7-b088-3844a18cc34e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6131b219fbf469bad78af5f21aaf4fe, user_id=e2ee317642aa4ddd9b90ce863d5d16e3, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:17.189517 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c69e203-154c-4df7-b088-3844a18cc34e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:17.212716 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1069/2145] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:17 2022] PUT /identity/v3/projects/d6131b219fbf469bad78af5f21aaf4fe/users/e2ee317642aa4ddd9b90ce863d5d16e3/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:17.225887 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5852ca6-d76f-40c9-8e5d-51b55ab7861a None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:17.226186 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5852ca6-d76f-40c9-8e5d-51b55ab7861a None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:17.226456 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5852ca6-d76f-40c9-8e5d-51b55ab7861a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:17.267899 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-e5852ca6-d76f-40c9-8e5d-51b55ab7861a None None] MFA Rules not processed for user `e2ee317642aa4ddd9b90ce863d5d16e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:17.269986 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-e5852ca6-d76f-40c9-8e5d-51b55ab7861a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:17.355578 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1077/2146] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:17.360852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c69e203-154c-4df7-b088-3844a18cc34e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:17.447095 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c69e203-154c-4df7-b088-3844a18cc34e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:17.452070 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ae9de-7c75-476b-a756-ba26bc999f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:17.452674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ae9de-7c75-476b-a756-ba26bc999f03 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:17.452979 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ae9de-7c75-476b-a756-ba26bc999f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:17.453257 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ae9de-7c75-476b-a756-ba26bc999f03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:17.453892 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ae9de-7c75-476b-a756-ba26bc999f03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:17.456689 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ae9de-7c75-476b-a756-ba26bc999f03 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:17.489758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1070/2147] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:42:17 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:42:17.490938 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5852ca6-d76f-40c9-8e5d-51b55ab7861a None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:17.510908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5852ca6-d76f-40c9-8e5d-51b55ab7861a None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:17.584900 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f52618-713c-41eb-a65b-8d50aedfe5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:17.586775 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f52618-713c-41eb-a65b-8d50aedfe5a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:17.586775 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f52618-713c-41eb-a65b-8d50aedfe5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:17.586775 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f52618-713c-41eb-a65b-8d50aedfe5a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:17.587076 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f52618-713c-41eb-a65b-8d50aedfe5a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:17.590297 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f52618-713c-41eb-a65b-8d50aedfe5a3 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:17.610218 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:17.622320 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1078/2148] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:42:17 2022] POST /identity/v3/users => generated 300 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:42:17.623414 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ae9de-7c75-476b-a756-ba26bc999f03 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:17.639088 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ae9de-7c75-476b-a756-ba26bc999f03 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:17.652826 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462095d-d948-47d7-bd11-49e98e2c4d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:17.655052 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462095d-d948-47d7-bd11-49e98e2c4d4b admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:17.656828 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462095d-d948-47d7-bd11-49e98e2c4d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:17.657778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462095d-d948-47d7-bd11-49e98e2c4d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:17.660167 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0462095d-d948-47d7-bd11-49e98e2c4d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:17.662262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0462095d-d948-47d7-bd11-49e98e2c4d4b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:17.691395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1071/2149] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:17.695448 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f52618-713c-41eb-a65b-8d50aedfe5a3 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:17.709477 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f52618-713c-41eb-a65b-8d50aedfe5a3 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:17.718233 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde6881-42d3-42f1-819f-801163ea3662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:17.718979 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde6881-42d3-42f1-819f-801163ea3662 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:17.719337 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde6881-42d3-42f1-819f-801163ea3662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:17.720155 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde6881-42d3-42f1-819f-801163ea3662 admin admin] PATH_INFO: `/v3/projects/a83f79d1481c454987df7db18adcb2aa/users/5534807503104949a7984e45c0953ee6/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:17.741139 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde6881-42d3-42f1-819f-801163ea3662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a83f79d1481c454987df7db18adcb2aa, user_id=5534807503104949a7984e45c0953ee6, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:17.745094 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde6881-42d3-42f1-819f-801163ea3662 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:17.768932 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1079/2150] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:17 2022] PUT /identity/v3/projects/a83f79d1481c454987df7db18adcb2aa/users/5534807503104949a7984e45c0953ee6/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:17.772488 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462095d-d948-47d7-bd11-49e98e2c4d4b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:17.805286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462095d-d948-47d7-bd11-49e98e2c4d4b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:17.836685 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:17.933099 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597bbd0-386f-41c0-ad72-8f6323d19427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:17.934072 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597bbd0-386f-41c0-ad72-8f6323d19427 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:17.934637 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597bbd0-386f-41c0-ad72-8f6323d19427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:17.935384 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597bbd0-386f-41c0-ad72-8f6323d19427 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:17.936737 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597bbd0-386f-41c0-ad72-8f6323d19427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:17.940056 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597bbd0-386f-41c0-ad72-8f6323d19427 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:17.979201 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1072/2151] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:42:17.981631 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde6881-42d3-42f1-819f-801163ea3662 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:18.008785 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde6881-42d3-42f1-819f-801163ea3662 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:18.020645 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8335cba-1723-44f8-a201-341dcf90ff0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:18.024149 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8335cba-1723-44f8-a201-341dcf90ff0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:18.026635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8335cba-1723-44f8-a201-341dcf90ff0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:18.027141 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8335cba-1723-44f8-a201-341dcf90ff0d admin admin] PATH_INFO: `/v3/projects/a83f79d1481c454987df7db18adcb2aa/users/5534807503104949a7984e45c0953ee6/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:18.032522 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8335cba-1723-44f8-a201-341dcf90ff0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a83f79d1481c454987df7db18adcb2aa, user_id=5534807503104949a7984e45c0953ee6, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:18.063849 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8335cba-1723-44f8-a201-341dcf90ff0d admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:18.109260 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1080/2152] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:17 2022] PUT /identity/v3/projects/a83f79d1481c454987df7db18adcb2aa/users/5534807503104949a7984e45c0953ee6/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:18.120130 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e47ae-b2ae-4e5d-bdb0-175652239aa1 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:18.121459 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e47ae-b2ae-4e5d-bdb0-175652239aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:18.121749 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e47ae-b2ae-4e5d-bdb0-175652239aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:18.176631 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e65e47ae-b2ae-4e5d-bdb0-175652239aa1 None None] MFA Rules not processed for user `5534807503104949a7984e45c0953ee6`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:18.178482 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e65e47ae-b2ae-4e5d-bdb0-175652239aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:18.318345 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8335cba-1723-44f8-a201-341dcf90ff0d admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:18.320509 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-d8335cba-1723-44f8-a201-341dcf90ff0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:18.347709 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1073/2153] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:18 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:18.362611 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e47ae-b2ae-4e5d-bdb0-175652239aa1 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:18.364172 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e65e47ae-b2ae-4e5d-bdb0-175652239aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:18.407409 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8335cba-1723-44f8-a201-341dcf90ff0d admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:18.412356 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180270e-fd37-4883-8e20-01455c48dbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:18.413023 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180270e-fd37-4883-8e20-01455c48dbae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:18.413344 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180270e-fd37-4883-8e20-01455c48dbae admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:18.413621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180270e-fd37-4883-8e20-01455c48dbae admin admin] PATH_INFO: `/v3/users/c9ac26a28e7b416b9786ddcacea5928e` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:18.414843 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180270e-fd37-4883-8e20-01455c48dbae admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9ac26a28e7b416b9786ddcacea5928e)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:18.416771 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180270e-fd37-4883-8e20-01455c48dbae admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:18.432631 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e47ae-b2ae-4e5d-bdb0-175652239aa1 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:18.436412 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f74375-fe57-4f18-82f9-fc70915f248f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:18.437092 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f74375-fe57-4f18-82f9-fc70915f248f service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:18.437403 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f74375-fe57-4f18-82f9-fc70915f248f service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:18.437653 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f74375-fe57-4f18-82f9-fc70915f248f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:18.448100 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f74375-fe57-4f18-82f9-fc70915f248f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:18.449667 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f74375-fe57-4f18-82f9-fc70915f248f service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:18.459935 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1074/2154] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:42:18 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:42:18.466794 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c180270e-fd37-4883-8e20-01455c48dbae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9ac26a28e7b416b9786ddcacea5928e'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:18.467479 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c180270e-fd37-4883-8e20-01455c48dbae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9ac26a28e7b416b9786ddcacea5928e'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:18.476459 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c180270e-fd37-4883-8e20-01455c48dbae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9ac26a28e7b416b9786ddcacea5928e'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:18.477216 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c180270e-fd37-4883-8e20-01455c48dbae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9ac26a28e7b416b9786ddcacea5928e'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:18.486006 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c180270e-fd37-4883-8e20-01455c48dbae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9ac26a28e7b416b9786ddcacea5928e'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:18.493967 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f74375-fe57-4f18-82f9-fc70915f248f service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:18.505464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f74375-fe57-4f18-82f9-fc70915f248f service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:18.511097 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb0e15-59ab-4da7-84e4-5abc154fcbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:18.511960 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb0e15-59ab-4da7-84e4-5abc154fcbe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:18.512316 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb0e15-59ab-4da7-84e4-5abc154fcbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:18.512916 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb0e15-59ab-4da7-84e4-5abc154fcbe0 admin admin] PATH_INFO: `/v3/users/e2ee317642aa4ddd9b90ce863d5d16e3` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:18.514286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cb0e15-59ab-4da7-84e4-5abc154fcbe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2ee317642aa4ddd9b90ce863d5d16e3)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:18.516758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cb0e15-59ab-4da7-84e4-5abc154fcbe0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:18.526001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1081/2155] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:42:18 2022] DELETE /identity/v3/users/c9ac26a28e7b416b9786ddcacea5928e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:18.581729 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:18.589383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e0cb0e15-59ab-4da7-84e4-5abc154fcbe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2ee317642aa4ddd9b90ce863d5d16e3'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:18.590425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e0cb0e15-59ab-4da7-84e4-5abc154fcbe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2ee317642aa4ddd9b90ce863d5d16e3'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:18.602279 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e0cb0e15-59ab-4da7-84e4-5abc154fcbe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2ee317642aa4ddd9b90ce863d5d16e3'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:18.612942 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e0cb0e15-59ab-4da7-84e4-5abc154fcbe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2ee317642aa4ddd9b90ce863d5d16e3'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:18.613631 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e0cb0e15-59ab-4da7-84e4-5abc154fcbe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2ee317642aa4ddd9b90ce863d5d16e3'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:18.637175 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1075/2156] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:42:18 2022] DELETE /identity/v3/users/e2ee317642aa4ddd9b90ce863d5d16e3 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:18.641981 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0ae4f-8450-41b1-8a00-d29562028675 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:18.642273 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0ae4f-8450-41b1-8a00-d29562028675 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:18.642550 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0ae4f-8450-41b1-8a00-d29562028675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:18.665672 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-b2e0ae4f-8450-41b1-8a00-d29562028675 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:18.667047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b2e0ae4f-8450-41b1-8a00-d29562028675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:18.708157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:18.725677 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb0e15-59ab-4da7-84e4-5abc154fcbe0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:18.728009 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e0cb0e15-59ab-4da7-84e4-5abc154fcbe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:18.749965 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1082/2157] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:18.761072 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0ae4f-8450-41b1-8a00-d29562028675 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:18.763003 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b2e0ae4f-8450-41b1-8a00-d29562028675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:18.816468 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb0e15-59ab-4da7-84e4-5abc154fcbe0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:18.821278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9facc-69cf-4ce8-8fff-61340b7cf080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:18.821916 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9facc-69cf-4ce8-8fff-61340b7cf080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:18.822200 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9facc-69cf-4ce8-8fff-61340b7cf080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:18.822469 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9facc-69cf-4ce8-8fff-61340b7cf080 admin admin] PATH_INFO: `/v3/projects/32ef0737a3844bfe8f4bc14b09ebc88b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:18.823625 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc9facc-69cf-4ce8-8fff-61340b7cf080 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32ef0737a3844bfe8f4bc14b09ebc88b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:18.825588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc9facc-69cf-4ce8-8fff-61340b7cf080 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:18.847638 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0ae4f-8450-41b1-8a00-d29562028675 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:18.851267 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca0ec5-a30d-4372-a055-2a239e9fa91e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:18.851883 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca0ec5-a30d-4372-a055-2a239e9fa91e service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:18.852150 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca0ec5-a30d-4372-a055-2a239e9fa91e service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:18.852424 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca0ec5-a30d-4372-a055-2a239e9fa91e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:18.863384 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca0ec5-a30d-4372-a055-2a239e9fa91e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:18.864950 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca0ec5-a30d-4372-a055-2a239e9fa91e service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:18.875045 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1083/2158] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:42:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:42:18.878317 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4cc9facc-69cf-4ce8-8fff-61340b7cf080 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32ef0737a3844bfe8f4bc14b09ebc88b', 'request_id': 'req-4cc9facc-69cf-4ce8-8fff-61340b7cf080'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:18.885206 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4cc9facc-69cf-4ce8-8fff-61340b7cf080 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32ef0737a3844bfe8f4bc14b09ebc88b', 'request_id': 'req-4cc9facc-69cf-4ce8-8fff-61340b7cf080'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:18.893984 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4cc9facc-69cf-4ce8-8fff-61340b7cf080 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32ef0737a3844bfe8f4bc14b09ebc88b', 'request_id': 'req-4cc9facc-69cf-4ce8-8fff-61340b7cf080'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:18.904836 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4cc9facc-69cf-4ce8-8fff-61340b7cf080 admin admin] The token cache is being invalidate because project 32ef0737a3844bfe8f4bc14b09ebc88b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:42:18.905262 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4cc9facc-69cf-4ce8-8fff-61340b7cf080 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:18.906049 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-4cc9facc-69cf-4ce8-8fff-61340b7cf080 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:18.907645 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1076/2159] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:42:18 2022] DELETE /identity/v3/projects/32ef0737a3844bfe8f4bc14b09ebc88b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:18.912543 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca0ec5-a30d-4372-a055-2a239e9fa91e service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:18.922696 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca0ec5-a30d-4372-a055-2a239e9fa91e service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:18.984811 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e41f8-d22f-4d1c-aa9e-0c9ef7e87e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:18.985467 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e41f8-d22f-4d1c-aa9e-0c9ef7e87e1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:18.985746 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e41f8-d22f-4d1c-aa9e-0c9ef7e87e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:18.986013 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e41f8-d22f-4d1c-aa9e-0c9ef7e87e1b admin admin] PATH_INFO: `/v3/users/87f47190e3234e0bb8cdb443e092e54f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:18.988296 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e41f8-d22f-4d1c-aa9e-0c9ef7e87e1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=87f47190e3234e0bb8cdb443e092e54f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:18.990263 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531e41f8-d22f-4d1c-aa9e-0c9ef7e87e1b admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:19.042159 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-531e41f8-d22f-4d1c-aa9e-0c9ef7e87e1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87f47190e3234e0bb8cdb443e092e54f'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:19.042974 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-531e41f8-d22f-4d1c-aa9e-0c9ef7e87e1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87f47190e3234e0bb8cdb443e092e54f'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:19.051253 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-531e41f8-d22f-4d1c-aa9e-0c9ef7e87e1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87f47190e3234e0bb8cdb443e092e54f'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:19.052064 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-531e41f8-d22f-4d1c-aa9e-0c9ef7e87e1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87f47190e3234e0bb8cdb443e092e54f'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:19.059986 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-531e41f8-d22f-4d1c-aa9e-0c9ef7e87e1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87f47190e3234e0bb8cdb443e092e54f'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:19.077936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1084/2160] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:42:18 2022] DELETE /identity/v3/users/87f47190e3234e0bb8cdb443e092e54f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:19.151144 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9facc-69cf-4ce8-8fff-61340b7cf080 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:19.152800 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4cc9facc-69cf-4ce8-8fff-61340b7cf080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:19.167694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9facc-69cf-4ce8-8fff-61340b7cf080 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:19.216576 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:19.232220 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecacfb88-e63c-4d90-a71d-e3ed97267ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:19.232824 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecacfb88-e63c-4d90-a71d-e3ed97267ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:19.233155 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecacfb88-e63c-4d90-a71d-e3ed97267ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:19.233578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecacfb88-e63c-4d90-a71d-e3ed97267ee5 admin admin] PATH_INFO: `/v3/projects/d6131b219fbf469bad78af5f21aaf4fe` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:19.234710 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecacfb88-e63c-4d90-a71d-e3ed97267ee5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6131b219fbf469bad78af5f21aaf4fe)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:19.236819 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecacfb88-e63c-4d90-a71d-e3ed97267ee5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:19.256857 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e41f8-d22f-4d1c-aa9e-0c9ef7e87e1b admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:19.258413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-531e41f8-d22f-4d1c-aa9e-0c9ef7e87e1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:19.269075 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:19.274625 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e41f8-d22f-4d1c-aa9e-0c9ef7e87e1b admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:19.281430 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f974a10-5b4c-48e3-a4a7-0b6e99c04baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:19.282404 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f974a10-5b4c-48e3-a4a7-0b6e99c04baa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:19.283199 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f974a10-5b4c-48e3-a4a7-0b6e99c04baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:19.285376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f974a10-5b4c-48e3-a4a7-0b6e99c04baa admin admin] PATH_INFO: `/v3/projects/111a918263d14198a3d24236431db365` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:19.287406 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f974a10-5b4c-48e3-a4a7-0b6e99c04baa admin admin] RBAC: Authorizing `identity:delete_project(project_id=111a918263d14198a3d24236431db365)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:19.290356 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f974a10-5b4c-48e3-a4a7-0b6e99c04baa admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:19.301666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ecacfb88-e63c-4d90-a71d-e3ed97267ee5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6131b219fbf469bad78af5f21aaf4fe', 'request_id': 'req-ecacfb88-e63c-4d90-a71d-e3ed97267ee5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:19.308139 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ecacfb88-e63c-4d90-a71d-e3ed97267ee5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6131b219fbf469bad78af5f21aaf4fe', 'request_id': 'req-ecacfb88-e63c-4d90-a71d-e3ed97267ee5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:19.316500 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ecacfb88-e63c-4d90-a71d-e3ed97267ee5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6131b219fbf469bad78af5f21aaf4fe', 'request_id': 'req-ecacfb88-e63c-4d90-a71d-e3ed97267ee5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:19.328082 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ecacfb88-e63c-4d90-a71d-e3ed97267ee5 admin admin] The token cache is being invalidate because project d6131b219fbf469bad78af5f21aaf4fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:42:19.328532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ecacfb88-e63c-4d90-a71d-e3ed97267ee5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:19.329272 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ecacfb88-e63c-4d90-a71d-e3ed97267ee5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:19.331632 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1077/2161] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:42:19 2022] DELETE /identity/v3/projects/d6131b219fbf469bad78af5f21aaf4fe => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:19.335668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecacfb88-e63c-4d90-a71d-e3ed97267ee5 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:19.337040 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ecacfb88-e63c-4d90-a71d-e3ed97267ee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:19.342956 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-7f974a10-5b4c-48e3-a4a7-0b6e99c04baa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '111a918263d14198a3d24236431db365', 'request_id': 'req-7f974a10-5b4c-48e3-a4a7-0b6e99c04baa'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:19.350217 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-7f974a10-5b4c-48e3-a4a7-0b6e99c04baa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '111a918263d14198a3d24236431db365', 'request_id': 'req-7f974a10-5b4c-48e3-a4a7-0b6e99c04baa'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:19.358267 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-7f974a10-5b4c-48e3-a4a7-0b6e99c04baa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '111a918263d14198a3d24236431db365', 'request_id': 'req-7f974a10-5b4c-48e3-a4a7-0b6e99c04baa'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:19.369941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-7f974a10-5b4c-48e3-a4a7-0b6e99c04baa admin admin] The token cache is being invalidate because project 111a918263d14198a3d24236431db365 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:42:19.370228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-7f974a10-5b4c-48e3-a4a7-0b6e99c04baa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:19.370866 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-7f974a10-5b4c-48e3-a4a7-0b6e99c04baa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:19.373348 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1085/2162] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:42:19 2022] DELETE /identity/v3/projects/111a918263d14198a3d24236431db365 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:19.403718 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecacfb88-e63c-4d90-a71d-e3ed97267ee5 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:19.480468 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c4cef-c8e9-488f-9a10-4702dc26edef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:19.481171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c4cef-c8e9-488f-9a10-4702dc26edef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:19.481541 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c4cef-c8e9-488f-9a10-4702dc26edef admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:19.481719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c4cef-c8e9-488f-9a10-4702dc26edef admin admin] PATH_INFO: `/v3/users/5534807503104949a7984e45c0953ee6` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:19.483042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c4cef-c8e9-488f-9a10-4702dc26edef admin admin] RBAC: Authorizing `identity:delete_user(user_id=5534807503104949a7984e45c0953ee6)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:19.485232 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c4cef-c8e9-488f-9a10-4702dc26edef admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:19.531539 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:19.571028 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e01c4cef-c8e9-488f-9a10-4702dc26edef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5534807503104949a7984e45c0953ee6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:19.571922 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e01c4cef-c8e9-488f-9a10-4702dc26edef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5534807503104949a7984e45c0953ee6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:19.584799 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e01c4cef-c8e9-488f-9a10-4702dc26edef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5534807503104949a7984e45c0953ee6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:19.601946 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e01c4cef-c8e9-488f-9a10-4702dc26edef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5534807503104949a7984e45c0953ee6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:19.602805 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-e01c4cef-c8e9-488f-9a10-4702dc26edef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5534807503104949a7984e45c0953ee6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:19.629695 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1078/2163] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:42:19 2022] DELETE /identity/v3/users/5534807503104949a7984e45c0953ee6 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:19.642779 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b9a23-c6c2-4cc9-a258-4c2dd1d4eff0 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:19.642887 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b9a23-c6c2-4cc9-a258-4c2dd1d4eff0 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:19.643754 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b9a23-c6c2-4cc9-a258-4c2dd1d4eff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:19.665763 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-f22b9a23-c6c2-4cc9-a258-4c2dd1d4eff0 None None] MFA Rules not processed for user `69c04718cd82496abbc82e353873e604`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:19.666986 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-f22b9a23-c6c2-4cc9-a258-4c2dd1d4eff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:19.748304 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1086/2164] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:19.939496 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c4cef-c8e9-488f-9a10-4702dc26edef admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:19.942232 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e01c4cef-c8e9-488f-9a10-4702dc26edef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:20.014610 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c4cef-c8e9-488f-9a10-4702dc26edef admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:20.019875 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ffd86d-7ac8-445f-975c-e105c72070a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:20.020509 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ffd86d-7ac8-445f-975c-e105c72070a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:20.020793 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ffd86d-7ac8-445f-975c-e105c72070a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:20.021299 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ffd86d-7ac8-445f-975c-e105c72070a1 admin admin] PATH_INFO: `/v3/projects/a83f79d1481c454987df7db18adcb2aa` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:20.022438 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ffd86d-7ac8-445f-975c-e105c72070a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a83f79d1481c454987df7db18adcb2aa)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:20.024346 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ffd86d-7ac8-445f-975c-e105c72070a1 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:20.074522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-68ffd86d-7ac8-445f-975c-e105c72070a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a83f79d1481c454987df7db18adcb2aa', 'request_id': 'req-68ffd86d-7ac8-445f-975c-e105c72070a1'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:20.080797 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-68ffd86d-7ac8-445f-975c-e105c72070a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a83f79d1481c454987df7db18adcb2aa', 'request_id': 'req-68ffd86d-7ac8-445f-975c-e105c72070a1'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:20.088548 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-68ffd86d-7ac8-445f-975c-e105c72070a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a83f79d1481c454987df7db18adcb2aa', 'request_id': 'req-68ffd86d-7ac8-445f-975c-e105c72070a1'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:20.099914 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-68ffd86d-7ac8-445f-975c-e105c72070a1 admin admin] The token cache is being invalidate because project a83f79d1481c454987df7db18adcb2aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:42:20.100253 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-68ffd86d-7ac8-445f-975c-e105c72070a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:20.100921 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-68ffd86d-7ac8-445f-975c-e105c72070a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:20.106050 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1079/2165] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:42:19 2022] DELETE /identity/v3/projects/a83f79d1481c454987df7db18adcb2aa => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:20.108500 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688ebf3-50ae-4a55-bf1e-20e830f32976 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:20.109002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688ebf3-50ae-4a55-bf1e-20e830f32976 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:20.109185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688ebf3-50ae-4a55-bf1e-20e830f32976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:20.129570 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-7688ebf3-50ae-4a55-bf1e-20e830f32976 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:20.130762 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-7688ebf3-50ae-4a55-bf1e-20e830f32976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:20.191291 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1087/2166] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:20.199122 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff68b6-9000-470c-86fb-1d1269e882d9 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:20.199424 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff68b6-9000-470c-86fb-1d1269e882d9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:20.199835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff68b6-9000-470c-86fb-1d1269e882d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:20.220979 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-8aff68b6-9000-470c-86fb-1d1269e882d9 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:20.222588 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8aff68b6-9000-470c-86fb-1d1269e882d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:20.230178 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1080/2167] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:20.234500 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688ebf3-50ae-4a55-bf1e-20e830f32976 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:20.244293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688ebf3-50ae-4a55-bf1e-20e830f32976 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:20.248891 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de4449-c9c8-450c-a121-5fb71ffd9cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:20.249438 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de4449-c9c8-450c-a121-5fb71ffd9cbd admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:20.249717 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de4449-c9c8-450c-a121-5fb71ffd9cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:20.249991 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de4449-c9c8-450c-a121-5fb71ffd9cbd admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:20.250656 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de4449-c9c8-450c-a121-5fb71ffd9cbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:42:20.250937 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de4449-c9c8-450c-a121-5fb71ffd9cbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:20.253763 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de4449-c9c8-450c-a121-5fb71ffd9cbd admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:20.266253 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1088/2168] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:42:20.270805 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff68b6-9000-470c-86fb-1d1269e882d9 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:20.280263 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff68b6-9000-470c-86fb-1d1269e882d9 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:20.285193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92ae7d-e0ac-4919-a56e-504e9ef622cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:20.285864 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92ae7d-e0ac-4919-a56e-504e9ef622cf admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:20.286144 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92ae7d-e0ac-4919-a56e-504e9ef622cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:20.286412 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92ae7d-e0ac-4919-a56e-504e9ef622cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:20.287045 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f92ae7d-e0ac-4919-a56e-504e9ef622cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:20.289016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f92ae7d-e0ac-4919-a56e-504e9ef622cf admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:20.323511 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1081/2169] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:42:20 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:42:20.325398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de4449-c9c8-450c-a121-5fb71ffd9cbd admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:20.339251 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de4449-c9c8-450c-a121-5fb71ffd9cbd admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:20.355458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:20.406664 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d904de53-cdee-42a5-bbee-7181f7b48a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:20.407268 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904de53-cdee-42a5-bbee-7181f7b48a4d admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:20.407546 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904de53-cdee-42a5-bbee-7181f7b48a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:20.407818 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904de53-cdee-42a5-bbee-7181f7b48a4d admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:20.408468 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d904de53-cdee-42a5-bbee-7181f7b48a4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:20.410669 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d904de53-cdee-42a5-bbee-7181f7b48a4d admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:20.432969 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1089/2170] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:42:20 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:42:20.437075 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92ae7d-e0ac-4919-a56e-504e9ef622cf admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:20.444524 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:20.448487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92ae7d-e0ac-4919-a56e-504e9ef622cf admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:20.452998 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d053d84-f8ee-4419-887c-413392bfd9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:20.453593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d053d84-f8ee-4419-887c-413392bfd9f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:20.453870 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d053d84-f8ee-4419-887c-413392bfd9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:20.454140 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d053d84-f8ee-4419-887c-413392bfd9f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:20.454933 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d053d84-f8ee-4419-887c-413392bfd9f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:20.456536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d053d84-f8ee-4419-887c-413392bfd9f3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:20.472104 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1082/2171] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:20.473305 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d904de53-cdee-42a5-bbee-7181f7b48a4d admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:20.483599 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d904de53-cdee-42a5-bbee-7181f7b48a4d admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:20.489087 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e712f3-de69-4dbb-b203-c286c7235f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:20.489686 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e712f3-de69-4dbb-b203-c286c7235f5b admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:20.489969 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e712f3-de69-4dbb-b203-c286c7235f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:20.490245 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e712f3-de69-4dbb-b203-c286c7235f5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:20.490887 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e712f3-de69-4dbb-b203-c286c7235f5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:20.492480 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e712f3-de69-4dbb-b203-c286c7235f5b admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:20.508159 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1090/2172] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:20.509264 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d053d84-f8ee-4419-887c-413392bfd9f3 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:20.520013 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d053d84-f8ee-4419-887c-413392bfd9f3 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:20.524723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fdb42-83d5-4862-ab33-aedecb95051d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:20.525476 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fdb42-83d5-4862-ab33-aedecb95051d admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:20.525772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fdb42-83d5-4862-ab33-aedecb95051d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:20.526023 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fdb42-83d5-4862-ab33-aedecb95051d admin admin] PATH_INFO: `/v3/projects/e39f8883f39f44b0b99444642534d9e4/users/95bec0e8da9e46fbaf6511953f9afced/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:20.541470 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5fdb42-83d5-4862-ab33-aedecb95051d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e39f8883f39f44b0b99444642534d9e4, user_id=95bec0e8da9e46fbaf6511953f9afced, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:20.544529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5fdb42-83d5-4862-ab33-aedecb95051d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:20.558733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1083/2173] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:20 2022] PUT /identity/v3/projects/e39f8883f39f44b0b99444642534d9e4/users/95bec0e8da9e46fbaf6511953f9afced/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:20.570007 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7a102-af89-4547-b01b-e24d2c9a37f6 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:20.570286 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7a102-af89-4547-b01b-e24d2c9a37f6 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:20.570557 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7a102-af89-4547-b01b-e24d2c9a37f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:20.599660 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-97d7a102-af89-4547-b01b-e24d2c9a37f6 None None] MFA Rules not processed for user `95bec0e8da9e46fbaf6511953f9afced`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:20.601316 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-97d7a102-af89-4547-b01b-e24d2c9a37f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:20.670864 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1091/2174] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:20.675647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fdb42-83d5-4862-ab33-aedecb95051d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:20.743911 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fdb42-83d5-4862-ab33-aedecb95051d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:20.748741 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c21321-4dcc-4d0f-b944-6eac4e089806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:20.749384 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c21321-4dcc-4d0f-b944-6eac4e089806 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:20.749664 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c21321-4dcc-4d0f-b944-6eac4e089806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:20.749937 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c21321-4dcc-4d0f-b944-6eac4e089806 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:20.750565 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c21321-4dcc-4d0f-b944-6eac4e089806 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:20.753438 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c21321-4dcc-4d0f-b944-6eac4e089806 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:20.785918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1084/2175] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:42:20 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:42:20.790013 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d7a102-af89-4547-b01b-e24d2c9a37f6 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:20.803238 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d7a102-af89-4547-b01b-e24d2c9a37f6 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:20.858775 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c21321-4dcc-4d0f-b944-6eac4e089806 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:20.860660 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-02c21321-4dcc-4d0f-b944-6eac4e089806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:20.876676 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08814a7d-7a22-45c1-82c8-fe7c44efba77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:20.878959 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08814a7d-7a22-45c1-82c8-fe7c44efba77 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:20.879322 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08814a7d-7a22-45c1-82c8-fe7c44efba77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:20.879698 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08814a7d-7a22-45c1-82c8-fe7c44efba77 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:20.880411 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08814a7d-7a22-45c1-82c8-fe7c44efba77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:20.882458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08814a7d-7a22-45c1-82c8-fe7c44efba77 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:20.913749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1092/2176] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:42:20 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:42:20.915345 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08814a7d-7a22-45c1-82c8-fe7c44efba77 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:20.926793 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08814a7d-7a22-45c1-82c8-fe7c44efba77 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:20.929990 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c21321-4dcc-4d0f-b944-6eac4e089806 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:20.931623 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74168480-eb6a-49a6-a729-733b2cc7d1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:20.932344 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74168480-eb6a-49a6-a729-733b2cc7d1ef admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:20.932624 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74168480-eb6a-49a6-a729-733b2cc7d1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:20.932924 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74168480-eb6a-49a6-a729-733b2cc7d1ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:20.933591 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74168480-eb6a-49a6-a729-733b2cc7d1ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:20.934351 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430477e6-fdc2-4984-bcd9-c14a0d252f58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:20.934989 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430477e6-fdc2-4984-bcd9-c14a0d252f58 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:20.935200 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74168480-eb6a-49a6-a729-733b2cc7d1ef admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:20.935303 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430477e6-fdc2-4984-bcd9-c14a0d252f58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:20.935558 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430477e6-fdc2-4984-bcd9-c14a0d252f58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:20.937503 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-430477e6-fdc2-4984-bcd9-c14a0d252f58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:20.946311 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1093/2177] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:42:20.950157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74168480-eb6a-49a6-a729-733b2cc7d1ef admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:20.952599 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430477e6-fdc2-4984-bcd9-c14a0d252f58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:20.954870 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430477e6-fdc2-4984-bcd9-c14a0d252f58 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:20.956369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:20.962713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74168480-eb6a-49a6-a729-733b2cc7d1ef admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:20.967223 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1085/2178] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:42:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:42:20.969169 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a4bec-fdec-4966-8c27-8045c5fb2e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:20.970150 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a4bec-fdec-4966-8c27-8045c5fb2e46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:20.970438 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a4bec-fdec-4966-8c27-8045c5fb2e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:20.970716 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a4bec-fdec-4966-8c27-8045c5fb2e46 admin admin] PATH_INFO: `/v3/projects/34ce3003287c4c919ccdce6144d1ff9a/users/cf67ce4a6cf24e06985d2b811a6e34e6/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:20.989449 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a4bec-fdec-4966-8c27-8045c5fb2e46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34ce3003287c4c919ccdce6144d1ff9a, user_id=cf67ce4a6cf24e06985d2b811a6e34e6, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:20.992131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a4bec-fdec-4966-8c27-8045c5fb2e46 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:21.007542 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1094/2179] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:20 2022] PUT /identity/v3/projects/34ce3003287c4c919ccdce6144d1ff9a/users/cf67ce4a6cf24e06985d2b811a6e34e6/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:21.011278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430477e6-fdc2-4984-bcd9-c14a0d252f58 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:21.023482 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430477e6-fdc2-4984-bcd9-c14a0d252f58 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:21.083944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:21.092524 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d7179-bbfb-427c-a690-3f13de591948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:21.093174 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d7179-bbfb-427c-a690-3f13de591948 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:21.093454 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d7179-bbfb-427c-a690-3f13de591948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:21.093727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d7179-bbfb-427c-a690-3f13de591948 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:21.094373 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5d7179-bbfb-427c-a690-3f13de591948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:21.096048 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5d7179-bbfb-427c-a690-3f13de591948 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:21.108893 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1086/2180] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:21.110246 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a4bec-fdec-4966-8c27-8045c5fb2e46 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:21.121533 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a4bec-fdec-4966-8c27-8045c5fb2e46 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:21.126910 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a106e2c-9c84-4247-961f-32706cfdf365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:21.128660 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a106e2c-9c84-4247-961f-32706cfdf365 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:21.128660 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a106e2c-9c84-4247-961f-32706cfdf365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:21.128913 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a106e2c-9c84-4247-961f-32706cfdf365 admin admin] PATH_INFO: `/v3/projects/34ce3003287c4c919ccdce6144d1ff9a/users/cf67ce4a6cf24e06985d2b811a6e34e6/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:21.131087 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a106e2c-9c84-4247-961f-32706cfdf365 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34ce3003287c4c919ccdce6144d1ff9a, user_id=cf67ce4a6cf24e06985d2b811a6e34e6, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:21.133805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a106e2c-9c84-4247-961f-32706cfdf365 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:21.144384 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1095/2181] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:21 2022] PUT /identity/v3/projects/34ce3003287c4c919ccdce6144d1ff9a/users/cf67ce4a6cf24e06985d2b811a6e34e6/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:21.155599 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c18a4-d007-446f-9bc3-8753dc7803fa None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:21.155888 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c18a4-d007-446f-9bc3-8753dc7803fa None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:21.156164 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c18a4-d007-446f-9bc3-8753dc7803fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:21.184613 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-6d1c18a4-d007-446f-9bc3-8753dc7803fa None None] MFA Rules not processed for user `cf67ce4a6cf24e06985d2b811a6e34e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:21.185865 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6d1c18a4-d007-446f-9bc3-8753dc7803fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:21.260390 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1087/2182] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:21.272178 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a106e2c-9c84-4247-961f-32706cfdf365 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:21.274115 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-6a106e2c-9c84-4247-961f-32706cfdf365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:21.336377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a106e2c-9c84-4247-961f-32706cfdf365 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:21.340098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4db68a-4f46-4c6a-adc7-6679bb972649 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:21.340719 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4db68a-4f46-4c6a-adc7-6679bb972649 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:21.341033 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4db68a-4f46-4c6a-adc7-6679bb972649 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:21.341308 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4db68a-4f46-4c6a-adc7-6679bb972649 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:21.352114 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4db68a-4f46-4c6a-adc7-6679bb972649 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:21.353789 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4db68a-4f46-4c6a-adc7-6679bb972649 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:21.364760 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1096/2183] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:42:21 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:42:21.460174 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c18a4-d007-446f-9bc3-8753dc7803fa None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:21.559929 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c18a4-d007-446f-9bc3-8753dc7803fa None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:21.579731 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8550a50-25dd-416d-8b5e-7d56bd45f99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:21.582681 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8550a50-25dd-416d-8b5e-7d56bd45f99e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:21.583208 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8550a50-25dd-416d-8b5e-7d56bd45f99e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:21.583552 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8550a50-25dd-416d-8b5e-7d56bd45f99e admin admin] PATH_INFO: `/v3/users/95bec0e8da9e46fbaf6511953f9afced` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:21.585180 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8550a50-25dd-416d-8b5e-7d56bd45f99e admin admin] RBAC: Authorizing `identity:delete_user(user_id=95bec0e8da9e46fbaf6511953f9afced)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:21.588218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8550a50-25dd-416d-8b5e-7d56bd45f99e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:21.656266 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b8550a50-25dd-416d-8b5e-7d56bd45f99e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95bec0e8da9e46fbaf6511953f9afced'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:21.659592 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b8550a50-25dd-416d-8b5e-7d56bd45f99e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95bec0e8da9e46fbaf6511953f9afced'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:21.670141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b8550a50-25dd-416d-8b5e-7d56bd45f99e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95bec0e8da9e46fbaf6511953f9afced'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:21.681033 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b8550a50-25dd-416d-8b5e-7d56bd45f99e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95bec0e8da9e46fbaf6511953f9afced'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:21.681712 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b8550a50-25dd-416d-8b5e-7d56bd45f99e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95bec0e8da9e46fbaf6511953f9afced'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:21.702735 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:21.709650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1088/2184] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:42:21 2022] DELETE /identity/v3/users/95bec0e8da9e46fbaf6511953f9afced => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:21.714214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b54c1-cada-4f48-933a-f68b7a12c543 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:21.714573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b54c1-cada-4f48-933a-f68b7a12c543 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:21.714849 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b54c1-cada-4f48-933a-f68b7a12c543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:21.734919 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-879b54c1-cada-4f48-933a-f68b7a12c543 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:21.736300 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-879b54c1-cada-4f48-933a-f68b7a12c543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:21.805820 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1097/2185] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:21.819407 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8550a50-25dd-416d-8b5e-7d56bd45f99e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:21.821414 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b8550a50-25dd-416d-8b5e-7d56bd45f99e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:21.892073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8550a50-25dd-416d-8b5e-7d56bd45f99e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:21.897130 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cadca-ed06-4b77-bb02-c878fd806399 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:21.897748 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cadca-ed06-4b77-bb02-c878fd806399 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:21.898030 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cadca-ed06-4b77-bb02-c878fd806399 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:21.898440 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cadca-ed06-4b77-bb02-c878fd806399 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:21.916131 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cadca-ed06-4b77-bb02-c878fd806399 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:21.917645 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cadca-ed06-4b77-bb02-c878fd806399 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:21.929216 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1089/2186] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:42:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:42:22.564239 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b54c1-cada-4f48-933a-f68b7a12c543 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:22.566218 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-879b54c1-cada-4f48-933a-f68b7a12c543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:22.579068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b54c1-cada-4f48-933a-f68b7a12c543 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:22.598714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32221d-2e39-475a-b2a4-c057ede73ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:22.608995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32221d-2e39-475a-b2a4-c057ede73ff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:22.618383 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32221d-2e39-475a-b2a4-c057ede73ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:22.618966 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32221d-2e39-475a-b2a4-c057ede73ff0 admin admin] PATH_INFO: `/v3/projects/e39f8883f39f44b0b99444642534d9e4` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:22.623797 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee32221d-2e39-475a-b2a4-c057ede73ff0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e39f8883f39f44b0b99444642534d9e4)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:22.625830 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee32221d-2e39-475a-b2a4-c057ede73ff0 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:22.730333 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ee32221d-2e39-475a-b2a4-c057ede73ff0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e39f8883f39f44b0b99444642534d9e4', 'request_id': 'req-ee32221d-2e39-475a-b2a4-c057ede73ff0'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:22.740608 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ee32221d-2e39-475a-b2a4-c057ede73ff0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e39f8883f39f44b0b99444642534d9e4', 'request_id': 'req-ee32221d-2e39-475a-b2a4-c057ede73ff0'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:22.754806 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ee32221d-2e39-475a-b2a4-c057ede73ff0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e39f8883f39f44b0b99444642534d9e4', 'request_id': 'req-ee32221d-2e39-475a-b2a4-c057ede73ff0'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:22.770866 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:22.786150 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ee32221d-2e39-475a-b2a4-c057ede73ff0 admin admin] The token cache is being invalidate because project e39f8883f39f44b0b99444642534d9e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:42:22.786792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ee32221d-2e39-475a-b2a4-c057ede73ff0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:22.787670 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ee32221d-2e39-475a-b2a4-c057ede73ff0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:22.789630 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1098/2187] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:42:22 2022] DELETE /identity/v3/projects/e39f8883f39f44b0b99444642534d9e4 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:22.796699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cadca-ed06-4b77-bb02-c878fd806399 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:22.812449 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cadca-ed06-4b77-bb02-c878fd806399 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:22.888583 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a831ed6-f773-4f3c-aef9-19f3858c1a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:22.889705 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a831ed6-f773-4f3c-aef9-19f3858c1a67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:22.890089 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a831ed6-f773-4f3c-aef9-19f3858c1a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:22.890461 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a831ed6-f773-4f3c-aef9-19f3858c1a67 admin admin] PATH_INFO: `/v3/users/cf67ce4a6cf24e06985d2b811a6e34e6` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:22.892067 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a831ed6-f773-4f3c-aef9-19f3858c1a67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf67ce4a6cf24e06985d2b811a6e34e6)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:22.894746 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a831ed6-f773-4f3c-aef9-19f3858c1a67 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:23.008105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1a831ed6-f773-4f3c-aef9-19f3858c1a67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf67ce4a6cf24e06985d2b811a6e34e6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:23.008784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1a831ed6-f773-4f3c-aef9-19f3858c1a67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf67ce4a6cf24e06985d2b811a6e34e6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:23.025526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1a831ed6-f773-4f3c-aef9-19f3858c1a67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf67ce4a6cf24e06985d2b811a6e34e6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:23.039001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1a831ed6-f773-4f3c-aef9-19f3858c1a67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf67ce4a6cf24e06985d2b811a6e34e6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:23.040320 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1a831ed6-f773-4f3c-aef9-19f3858c1a67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf67ce4a6cf24e06985d2b811a6e34e6'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:23.080113 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1090/2188] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:42:22 2022] DELETE /identity/v3/users/cf67ce4a6cf24e06985d2b811a6e34e6 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:23.756194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32221d-2e39-475a-b2a4-c057ede73ff0 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:23.757689 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-ee32221d-2e39-475a-b2a4-c057ede73ff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:23.784318 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32221d-2e39-475a-b2a4-c057ede73ff0 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:23.872497 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0099fe5-64b8-4b42-992f-a5760a5f5d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:23.873213 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0099fe5-64b8-4b42-992f-a5760a5f5d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:23.873550 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0099fe5-64b8-4b42-992f-a5760a5f5d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:23.873830 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0099fe5-64b8-4b42-992f-a5760a5f5d3d admin admin] PATH_INFO: `/v3/projects/34ce3003287c4c919ccdce6144d1ff9a` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:23.874999 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0099fe5-64b8-4b42-992f-a5760a5f5d3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=34ce3003287c4c919ccdce6144d1ff9a)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:23.877673 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0099fe5-64b8-4b42-992f-a5760a5f5d3d admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:23.943688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c0099fe5-64b8-4b42-992f-a5760a5f5d3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34ce3003287c4c919ccdce6144d1ff9a', 'request_id': 'req-c0099fe5-64b8-4b42-992f-a5760a5f5d3d'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:23.957623 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c0099fe5-64b8-4b42-992f-a5760a5f5d3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34ce3003287c4c919ccdce6144d1ff9a', 'request_id': 'req-c0099fe5-64b8-4b42-992f-a5760a5f5d3d'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:23.967910 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c0099fe5-64b8-4b42-992f-a5760a5f5d3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34ce3003287c4c919ccdce6144d1ff9a', 'request_id': 'req-c0099fe5-64b8-4b42-992f-a5760a5f5d3d'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:23.982392 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c0099fe5-64b8-4b42-992f-a5760a5f5d3d admin admin] The token cache is being invalidate because project 34ce3003287c4c919ccdce6144d1ff9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:42:23.983067 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c0099fe5-64b8-4b42-992f-a5760a5f5d3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:23.984102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c0099fe5-64b8-4b42-992f-a5760a5f5d3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:23.986362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1099/2189] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:42:23 2022] DELETE /identity/v3/projects/34ce3003287c4c919ccdce6144d1ff9a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:23.991548 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e0ed8-f0d0-4c9a-91fa-ab46a35079f0 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:23.993310 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e0ed8-f0d0-4c9a-91fa-ab46a35079f0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:23.993855 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e0ed8-f0d0-4c9a-91fa-ab46a35079f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:24.015221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-953e0ed8-f0d0-4c9a-91fa-ab46a35079f0 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:24.016896 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-953e0ed8-f0d0-4c9a-91fa-ab46a35079f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:24.034934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:24.142676 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1091/2190] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:24.150674 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1089515-2d74-41d4-b64b-2ecad2d98aa0 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:24.150964 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1089515-2d74-41d4-b64b-2ecad2d98aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:24.151240 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1089515-2d74-41d4-b64b-2ecad2d98aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:24.180195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-c1089515-2d74-41d4-b64b-2ecad2d98aa0 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:24.183309 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-c1089515-2d74-41d4-b64b-2ecad2d98aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:24.195968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1100/2191] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:24.200288 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e0ed8-f0d0-4c9a-91fa-ab46a35079f0 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:24.216124 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e0ed8-f0d0-4c9a-91fa-ab46a35079f0 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:24.235765 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cc269-8a22-4744-888e-a8746d15451c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:24.235765 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cc269-8a22-4744-888e-a8746d15451c admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:24.235765 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cc269-8a22-4744-888e-a8746d15451c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:24.235765 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cc269-8a22-4744-888e-a8746d15451c admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:24.236321 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cc269-8a22-4744-888e-a8746d15451c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:42:24.236321 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cc269-8a22-4744-888e-a8746d15451c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:24.236321 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cc269-8a22-4744-888e-a8746d15451c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:24.239598 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1092/2192] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:42:24.244269 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1089515-2d74-41d4-b64b-2ecad2d98aa0 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:24.263854 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1089515-2d74-41d4-b64b-2ecad2d98aa0 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:24.271076 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af35dd-3f5d-40ac-9b46-282dba193450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:24.271818 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af35dd-3f5d-40ac-9b46-282dba193450 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:24.272228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af35dd-3f5d-40ac-9b46-282dba193450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:24.272568 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af35dd-3f5d-40ac-9b46-282dba193450 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:24.273836 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af35dd-3f5d-40ac-9b46-282dba193450 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:24.275843 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af35dd-3f5d-40ac-9b46-282dba193450 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:24.310766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1101/2193] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:42:24 2022] POST /identity/v3/projects => generated 381 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:42:24.312330 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cc269-8a22-4744-888e-a8746d15451c admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:24.323423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cc269-8a22-4744-888e-a8746d15451c admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:24.385047 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642ac3f-677c-459c-8201-dc12c25ebf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:24.385654 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642ac3f-677c-459c-8201-dc12c25ebf28 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:24.385931 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642ac3f-677c-459c-8201-dc12c25ebf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:24.386205 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642ac3f-677c-459c-8201-dc12c25ebf28 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:24.386830 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642ac3f-677c-459c-8201-dc12c25ebf28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:24.388732 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642ac3f-677c-459c-8201-dc12c25ebf28 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:24.415456 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1093/2194] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:42:24 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:42:24.416590 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af35dd-3f5d-40ac-9b46-282dba193450 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:24.427700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af35dd-3f5d-40ac-9b46-282dba193450 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:24.432450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a70aa1-eb61-4adb-a6d3-4193f7ef6f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:24.433167 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a70aa1-eb61-4adb-a6d3-4193f7ef6f4b admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:24.433497 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a70aa1-eb61-4adb-a6d3-4193f7ef6f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:24.433862 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a70aa1-eb61-4adb-a6d3-4193f7ef6f4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:24.434589 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a70aa1-eb61-4adb-a6d3-4193f7ef6f4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:24.436259 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a70aa1-eb61-4adb-a6d3-4193f7ef6f4b admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:24.457710 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1102/2195] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:42:24.461762 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642ac3f-677c-459c-8201-dc12c25ebf28 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:24.472206 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642ac3f-677c-459c-8201-dc12c25ebf28 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:24.477157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e65a5-d846-4b99-9d4c-e11cdf4431e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:24.477832 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e65a5-d846-4b99-9d4c-e11cdf4431e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:24.478175 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e65a5-d846-4b99-9d4c-e11cdf4431e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:24.478507 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e65a5-d846-4b99-9d4c-e11cdf4431e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:24.479225 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4e65a5-d846-4b99-9d4c-e11cdf4431e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:24.481033 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4e65a5-d846-4b99-9d4c-e11cdf4431e8 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:24.492920 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1094/2196] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:24.498012 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a70aa1-eb61-4adb-a6d3-4193f7ef6f4b admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:24.517187 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a70aa1-eb61-4adb-a6d3-4193f7ef6f4b admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:24.522939 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3b824-b783-463d-adef-6ae66a74c155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:24.523607 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3b824-b783-463d-adef-6ae66a74c155 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:24.523886 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3b824-b783-463d-adef-6ae66a74c155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:24.524158 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3b824-b783-463d-adef-6ae66a74c155 admin admin] PATH_INFO: `/v3/projects/69d8f8cde4de4e269ec5e2858835b8a5/users/9dfc0bfb4a4f40929afa66251820f1d3/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:24.542117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b3b824-b783-463d-adef-6ae66a74c155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69d8f8cde4de4e269ec5e2858835b8a5, user_id=9dfc0bfb4a4f40929afa66251820f1d3, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:24.545229 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b3b824-b783-463d-adef-6ae66a74c155 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:24.566812 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1103/2197] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:24 2022] PUT /identity/v3/projects/69d8f8cde4de4e269ec5e2858835b8a5/users/9dfc0bfb4a4f40929afa66251820f1d3/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:24.577706 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d25ad-d55e-466f-ae5e-c366a3873d0a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:24.577993 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d25ad-d55e-466f-ae5e-c366a3873d0a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:24.578268 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d25ad-d55e-466f-ae5e-c366a3873d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:24.623642 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2d8d25ad-d55e-466f-ae5e-c366a3873d0a None None] MFA Rules not processed for user `9dfc0bfb4a4f40929afa66251820f1d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:24.627829 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2d8d25ad-d55e-466f-ae5e-c366a3873d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:24.734125 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1095/2198] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:24.740161 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3b824-b783-463d-adef-6ae66a74c155 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:24.771039 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:24.824364 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3b824-b783-463d-adef-6ae66a74c155 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:24.835161 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b3a7f-44e6-4d4c-b2a3-ef9b3b45d728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:24.835798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b3a7f-44e6-4d4c-b2a3-ef9b3b45d728 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:24.836080 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b3a7f-44e6-4d4c-b2a3-ef9b3b45d728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:24.836415 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b3a7f-44e6-4d4c-b2a3-ef9b3b45d728 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:24.837130 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b3a7f-44e6-4d4c-b2a3-ef9b3b45d728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:24.839096 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b3a7f-44e6-4d4c-b2a3-ef9b3b45d728 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:24.881535 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1104/2199] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:42:24 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:42:24.882426 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d25ad-d55e-466f-ae5e-c366a3873d0a None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:24.902283 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d25ad-d55e-466f-ae5e-c366a3873d0a None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:24.927035 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:24.985642 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080b3e9-dce3-4a65-b359-043389571b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:24.986267 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080b3e9-dce3-4a65-b359-043389571b0b admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:24.987508 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080b3e9-dce3-4a65-b359-043389571b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:24.987874 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080b3e9-dce3-4a65-b359-043389571b0b admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:24.988529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080b3e9-dce3-4a65-b359-043389571b0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:24.990560 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080b3e9-dce3-4a65-b359-043389571b0b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:25.019402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1096/2200] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:42:24 2022] POST /identity/v3/users => generated 296 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:42:25.020579 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b3a7f-44e6-4d4c-b2a3-ef9b3b45d728 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:25.031651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b3a7f-44e6-4d4c-b2a3-ef9b3b45d728 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:25.037160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9d4c6-008f-48fc-a8c8-cd18a9dfb046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:25.037832 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9d4c6-008f-48fc-a8c8-cd18a9dfb046 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:25.038197 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9d4c6-008f-48fc-a8c8-cd18a9dfb046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:25.038557 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9d4c6-008f-48fc-a8c8-cd18a9dfb046 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:25.039266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc9d4c6-008f-48fc-a8c8-cd18a9dfb046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:25.040877 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc9d4c6-008f-48fc-a8c8-cd18a9dfb046 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:25.053348 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1105/2201] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:25.057663 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080b3e9-dce3-4a65-b359-043389571b0b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:25.068858 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080b3e9-dce3-4a65-b359-043389571b0b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:25.074051 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e3e5d-0d54-4ea8-8157-3a7af29d4b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:25.074854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e3e5d-0d54-4ea8-8157-3a7af29d4b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:25.075134 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e3e5d-0d54-4ea8-8157-3a7af29d4b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:25.075409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e3e5d-0d54-4ea8-8157-3a7af29d4b50 admin admin] PATH_INFO: `/v3/projects/72ccc6add928467da2c36f2bbedc0b52/users/a0f546b4cdd74c459f77b22f183d8725/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:25.091059 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871e3e5d-0d54-4ea8-8157-3a7af29d4b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72ccc6add928467da2c36f2bbedc0b52, user_id=a0f546b4cdd74c459f77b22f183d8725, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:25.093823 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871e3e5d-0d54-4ea8-8157-3a7af29d4b50 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:25.110632 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1097/2202] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:25 2022] PUT /identity/v3/projects/72ccc6add928467da2c36f2bbedc0b52/users/a0f546b4cdd74c459f77b22f183d8725/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:25.115300 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9d4c6-008f-48fc-a8c8-cd18a9dfb046 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:25.125632 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9d4c6-008f-48fc-a8c8-cd18a9dfb046 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:25.199778 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b38cb-b016-4142-b171-b02a539eda45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:25.200446 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b38cb-b016-4142-b171-b02a539eda45 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:25.200735 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b38cb-b016-4142-b171-b02a539eda45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:25.201038 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b38cb-b016-4142-b171-b02a539eda45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:25.201677 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b38cb-b016-4142-b171-b02a539eda45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:25.203334 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b38cb-b016-4142-b171-b02a539eda45 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:25.217746 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1106/2203] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:42:25.218996 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e3e5d-0d54-4ea8-8157-3a7af29d4b50 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:25.229992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e3e5d-0d54-4ea8-8157-3a7af29d4b50 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:25.234687 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee6684-026b-4fbf-b374-f660d3952cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:25.235314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee6684-026b-4fbf-b374-f660d3952cbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:25.235598 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee6684-026b-4fbf-b374-f660d3952cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:25.235878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee6684-026b-4fbf-b374-f660d3952cbf admin admin] PATH_INFO: `/v3/projects/72ccc6add928467da2c36f2bbedc0b52/users/a0f546b4cdd74c459f77b22f183d8725/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:25.240121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee6684-026b-4fbf-b374-f660d3952cbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=72ccc6add928467da2c36f2bbedc0b52, user_id=a0f546b4cdd74c459f77b22f183d8725, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:25.241116 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee6684-026b-4fbf-b374-f660d3952cbf admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:25.260947 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1098/2204] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:25 2022] PUT /identity/v3/projects/72ccc6add928467da2c36f2bbedc0b52/users/a0f546b4cdd74c459f77b22f183d8725/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:25.268764 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedabdaf-d881-43b0-9a88-6e0212de8e24 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:25.269259 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedabdaf-d881-43b0-9a88-6e0212de8e24 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:25.269545 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedabdaf-d881-43b0-9a88-6e0212de8e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:25.275953 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee6684-026b-4fbf-b374-f660d3952cbf admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:25.277804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-16ee6684-026b-4fbf-b374-f660d3952cbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:25.306385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-bedabdaf-d881-43b0-9a88-6e0212de8e24 None None] MFA Rules not processed for user `a0f546b4cdd74c459f77b22f183d8725`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:25.308788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-bedabdaf-d881-43b0-9a88-6e0212de8e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:25.361332 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee6684-026b-4fbf-b374-f660d3952cbf admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:25.367563 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6573493-bec3-4c1d-a4f9-66c5e15c002d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:25.368329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6573493-bec3-4c1d-a4f9-66c5e15c002d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:25.368885 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6573493-bec3-4c1d-a4f9-66c5e15c002d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:25.369180 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6573493-bec3-4c1d-a4f9-66c5e15c002d admin admin] PATH_INFO: `/v3/users/144e9801ac1f470083d11647652ccdf1` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:25.370558 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6573493-bec3-4c1d-a4f9-66c5e15c002d admin admin] RBAC: Authorizing `identity:delete_user(user_id=144e9801ac1f470083d11647652ccdf1)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:25.372484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6573493-bec3-4c1d-a4f9-66c5e15c002d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:25.398133 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1107/2205] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:25.409563 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedabdaf-d881-43b0-9a88-6e0212de8e24 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:25.411040 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-bedabdaf-d881-43b0-9a88-6e0212de8e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:25.432143 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b6573493-bec3-4c1d-a4f9-66c5e15c002d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '144e9801ac1f470083d11647652ccdf1'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:25.432718 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:25.433294 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b6573493-bec3-4c1d-a4f9-66c5e15c002d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '144e9801ac1f470083d11647652ccdf1'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:25.442891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b6573493-bec3-4c1d-a4f9-66c5e15c002d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '144e9801ac1f470083d11647652ccdf1'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:25.452173 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b6573493-bec3-4c1d-a4f9-66c5e15c002d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '144e9801ac1f470083d11647652ccdf1'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:25.453337 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b6573493-bec3-4c1d-a4f9-66c5e15c002d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '144e9801ac1f470083d11647652ccdf1'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:25.461921 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=85,keystone:INSERT=6,keystone:DELETE=9 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:25.476906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1099/2206] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:42:25 2022] DELETE /identity/v3/users/144e9801ac1f470083d11647652ccdf1 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:25.485221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedabdaf-d881-43b0-9a88-6e0212de8e24 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:25.561487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb896a0-b018-407c-8d46-be8a159843e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:25.562095 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb896a0-b018-407c-8d46-be8a159843e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:25.562371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb896a0-b018-407c-8d46-be8a159843e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:25.562641 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb896a0-b018-407c-8d46-be8a159843e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:25.565126 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-6fb896a0-b018-407c-8d46-be8a159843e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:25.631338 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb896a0-b018-407c-8d46-be8a159843e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:25.632847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb896a0-b018-407c-8d46-be8a159843e0 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:25.642556 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1108/2207] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:42:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:42:25.682248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6573493-bec3-4c1d-a4f9-66c5e15c002d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:25.683850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b6573493-bec3-4c1d-a4f9-66c5e15c002d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:25.772883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6573493-bec3-4c1d-a4f9-66c5e15c002d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:25.777518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c924d358-d15e-4ed6-8f6a-f071f8332234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:25.778256 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924d358-d15e-4ed6-8f6a-f071f8332234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:25.778727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924d358-d15e-4ed6-8f6a-f071f8332234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:25.779087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924d358-d15e-4ed6-8f6a-f071f8332234 admin admin] PATH_INFO: `/v3/projects/b215cb88123a4851932dbf7dfa88f709` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:25.780336 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c924d358-d15e-4ed6-8f6a-f071f8332234 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b215cb88123a4851932dbf7dfa88f709)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:25.782362 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c924d358-d15e-4ed6-8f6a-f071f8332234 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:25.834873 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c924d358-d15e-4ed6-8f6a-f071f8332234 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b215cb88123a4851932dbf7dfa88f709', 'request_id': 'req-c924d358-d15e-4ed6-8f6a-f071f8332234'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:25.842164 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c924d358-d15e-4ed6-8f6a-f071f8332234 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b215cb88123a4851932dbf7dfa88f709', 'request_id': 'req-c924d358-d15e-4ed6-8f6a-f071f8332234'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:25.851521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c924d358-d15e-4ed6-8f6a-f071f8332234 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b215cb88123a4851932dbf7dfa88f709', 'request_id': 'req-c924d358-d15e-4ed6-8f6a-f071f8332234'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:25.864102 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c924d358-d15e-4ed6-8f6a-f071f8332234 admin admin] The token cache is being invalidate because project b215cb88123a4851932dbf7dfa88f709 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:42:25.864395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c924d358-d15e-4ed6-8f6a-f071f8332234 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:25.865074 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-c924d358-d15e-4ed6-8f6a-f071f8332234 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:25.867079 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1100/2208] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:42:25 2022] DELETE /identity/v3/projects/b215cb88123a4851932dbf7dfa88f709 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:25.871914 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa3133-d77f-473b-b042-30398017af18 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:25.872381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa3133-d77f-473b-b042-30398017af18 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:25.872519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa3133-d77f-473b-b042-30398017af18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:25.893817 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-c2aa3133-d77f-473b-b042-30398017af18 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:25.895213 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-c2aa3133-d77f-473b-b042-30398017af18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:25.965072 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1109/2209] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:42:25.973715 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20920b-0cc7-4b28-8ea2-47779e85a9d5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:25.974006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20920b-0cc7-4b28-8ea2-47779e85a9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:25.974303 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20920b-0cc7-4b28-8ea2-47779e85a9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:25.996239 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0d20920b-0cc7-4b28-8ea2-47779e85a9d5 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:25.997473 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0d20920b-0cc7-4b28-8ea2-47779e85a9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:26.006704 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1101/2210] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:26.010890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa3133-d77f-473b-b042-30398017af18 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:26.022390 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa3133-d77f-473b-b042-30398017af18 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:26.028504 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f05cd-515f-4972-9151-b2c8f136f990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:26.029246 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f05cd-515f-4972-9151-b2c8f136f990 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:26.029495 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f05cd-515f-4972-9151-b2c8f136f990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:26.029768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f05cd-515f-4972-9151-b2c8f136f990 admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:26.030425 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f05cd-515f-4972-9151-b2c8f136f990 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:42:26.030728 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f05cd-515f-4972-9151-b2c8f136f990 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:26.032359 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f05cd-515f-4972-9151-b2c8f136f990 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:26.045911 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20920b-0cc7-4b28-8ea2-47779e85a9d5 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:26.047350 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0d20920b-0cc7-4b28-8ea2-47779e85a9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:26.048221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1110/2211] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:42:26.052544 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f05cd-515f-4972-9151-b2c8f136f990 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:26.063636 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f05cd-515f-4972-9151-b2c8f136f990 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:26.065039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20920b-0cc7-4b28-8ea2-47779e85a9d5 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:26.069180 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31f9c3-1f20-410a-81f0-e079b90e293b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:26.069762 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31f9c3-1f20-410a-81f0-e079b90e293b admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:26.069989 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7a703-baf4-4689-af81-cdbcd6f55b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:26.070129 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31f9c3-1f20-410a-81f0-e079b90e293b admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:26.070296 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31f9c3-1f20-410a-81f0-e079b90e293b admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:26.070788 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7a703-baf4-4689-af81-cdbcd6f55b8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:26.070977 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31f9c3-1f20-410a-81f0-e079b90e293b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:26.071178 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7a703-baf4-4689-af81-cdbcd6f55b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:26.071511 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7a703-baf4-4689-af81-cdbcd6f55b8f admin admin] PATH_INFO: `/v3/users/9dfc0bfb4a4f40929afa66251820f1d3` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:26.072706 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7a703-baf4-4689-af81-cdbcd6f55b8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dfc0bfb4a4f40929afa66251820f1d3)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:26.073059 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31f9c3-1f20-410a-81f0-e079b90e293b admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:26.074760 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7a703-baf4-4689-af81-cdbcd6f55b8f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:26.079315 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:26.103528 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1111/2212] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:42:26 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:42:26.108220 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31f9c3-1f20-410a-81f0-e079b90e293b admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:26.133287 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9bf7a703-baf4-4689-af81-cdbcd6f55b8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dfc0bfb4a4f40929afa66251820f1d3'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:26.133963 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9bf7a703-baf4-4689-af81-cdbcd6f55b8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dfc0bfb4a4f40929afa66251820f1d3'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:26.141722 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9bf7a703-baf4-4689-af81-cdbcd6f55b8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dfc0bfb4a4f40929afa66251820f1d3'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:26.150130 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9bf7a703-baf4-4689-af81-cdbcd6f55b8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dfc0bfb4a4f40929afa66251820f1d3'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:26.150891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-9bf7a703-baf4-4689-af81-cdbcd6f55b8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dfc0bfb4a4f40929afa66251820f1d3'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:26.153328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:26.172441 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1102/2213] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:42:26 2022] DELETE /identity/v3/users/9dfc0bfb4a4f40929afa66251820f1d3 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:26.177117 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31f9c3-1f20-410a-81f0-e079b90e293b admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:26.177379 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445e831-b3f1-4117-a0da-6bd9f136ab36 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:26.177766 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445e831-b3f1-4117-a0da-6bd9f136ab36 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:26.178144 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445e831-b3f1-4117-a0da-6bd9f136ab36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:26.199519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-e445e831-b3f1-4117-a0da-6bd9f136ab36 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:26.200646 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e445e831-b3f1-4117-a0da-6bd9f136ab36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:26.236604 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7cedb4-2546-417a-898f-0332a9bd05fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:26.237235 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7cedb4-2546-417a-898f-0332a9bd05fa admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:26.237517 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7cedb4-2546-417a-898f-0332a9bd05fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:26.237792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7cedb4-2546-417a-898f-0332a9bd05fa admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:26.238422 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7cedb4-2546-417a-898f-0332a9bd05fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:26.240351 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7cedb4-2546-417a-898f-0332a9bd05fa admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:26.260964 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1103/2214] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:42:26.265512 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1112/2215] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:42:26 2022] POST /identity/v3/users => generated 283 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:42:26.270446 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445e831-b3f1-4117-a0da-6bd9f136ab36 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:26.271699 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7cedb4-2546-417a-898f-0332a9bd05fa admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:26.272059 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e445e831-b3f1-4117-a0da-6bd9f136ab36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:26.273357 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-df7cedb4-2546-417a-898f-0332a9bd05fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:26.283833 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445e831-b3f1-4117-a0da-6bd9f136ab36 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:26.289072 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b092221-9f95-4dd0-bf9b-93744aabf01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:26.289680 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b092221-9f95-4dd0-bf9b-93744aabf01d admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:26.289961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b092221-9f95-4dd0-bf9b-93744aabf01d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:26.290238 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b092221-9f95-4dd0-bf9b-93744aabf01d admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:26.290891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b092221-9f95-4dd0-bf9b-93744aabf01d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:26.292539 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b092221-9f95-4dd0-bf9b-93744aabf01d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:26.303308 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1104/2216] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:42:26.307356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b092221-9f95-4dd0-bf9b-93744aabf01d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:26.317924 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b092221-9f95-4dd0-bf9b-93744aabf01d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:26.323584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711fa426-99de-4f79-b8d9-23b8f453f436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:26.323886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fa426-99de-4f79-b8d9-23b8f453f436 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:26.324168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fa426-99de-4f79-b8d9-23b8f453f436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:26.324460 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711fa426-99de-4f79-b8d9-23b8f453f436 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:26.325117 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711fa426-99de-4f79-b8d9-23b8f453f436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:26.326755 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711fa426-99de-4f79-b8d9-23b8f453f436 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:26.334047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7cedb4-2546-417a-898f-0332a9bd05fa admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:26.337069 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1105/2217] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:42:26.337859 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246dde5-b563-4285-825b-730d3c3dc48c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:26.338468 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246dde5-b563-4285-825b-730d3c3dc48c service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:26.338746 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246dde5-b563-4285-825b-730d3c3dc48c service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:26.339016 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246dde5-b563-4285-825b-730d3c3dc48c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:26.341094 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711fa426-99de-4f79-b8d9-23b8f453f436 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:26.348340 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246dde5-b563-4285-825b-730d3c3dc48c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:26.349873 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246dde5-b563-4285-825b-730d3c3dc48c service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:26.351093 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711fa426-99de-4f79-b8d9-23b8f453f436 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:26.355692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbcff61-422d-422c-b476-ffbd2be28c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:26.356324 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcff61-422d-422c-b476-ffbd2be28c8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:26.356603 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcff61-422d-422c-b476-ffbd2be28c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:26.356901 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcff61-422d-422c-b476-ffbd2be28c8e admin admin] PATH_INFO: `/v3/projects/27a8f2a584c4414b89c877dd143c6207/users/3b1ef7d93a474cbf96de059d535a9cdf/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:26.359245 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1113/2218] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:42:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:42:26.373374 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbcff61-422d-422c-b476-ffbd2be28c8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=27a8f2a584c4414b89c877dd143c6207, user_id=3b1ef7d93a474cbf96de059d535a9cdf, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:26.376067 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbcff61-422d-422c-b476-ffbd2be28c8e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:26.391508 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1106/2219] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:26 2022] PUT /identity/v3/projects/27a8f2a584c4414b89c877dd143c6207/users/3b1ef7d93a474cbf96de059d535a9cdf/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:26.401874 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8224cb-da9d-44ab-ba1a-9f80fd1ec5d5 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:26.402160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8224cb-da9d-44ab-ba1a-9f80fd1ec5d5 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:26.402435 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8224cb-da9d-44ab-ba1a-9f80fd1ec5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:26.433646 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-6f8224cb-da9d-44ab-ba1a-9f80fd1ec5d5 None None] MFA Rules not processed for user `3b1ef7d93a474cbf96de059d535a9cdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:26.434846 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-6f8224cb-da9d-44ab-ba1a-9f80fd1ec5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:26.448544 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:26.506046 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1114/2220] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:26.510611 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbcff61-422d-422c-b476-ffbd2be28c8e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:26.589181 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbcff61-422d-422c-b476-ffbd2be28c8e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:26.594101 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b5d22-0ea1-4162-a6ef-5ce1552fdbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:26.595092 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b5d22-0ea1-4162-a6ef-5ce1552fdbe1 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:26.595092 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b5d22-0ea1-4162-a6ef-5ce1552fdbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:26.595276 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b5d22-0ea1-4162-a6ef-5ce1552fdbe1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:26.596538 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b5d22-0ea1-4162-a6ef-5ce1552fdbe1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:26.598477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b5d22-0ea1-4162-a6ef-5ce1552fdbe1 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:26.630355 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1107/2221] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:42:26 2022] POST /identity/v3/projects => generated 363 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:42:26.634510 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8224cb-da9d-44ab-ba1a-9f80fd1ec5d5 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:26.646751 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8224cb-da9d-44ab-ba1a-9f80fd1ec5d5 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:26.671608 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b5d22-0ea1-4162-a6ef-5ce1552fdbe1 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:26.676655 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e33b5d22-0ea1-4162-a6ef-5ce1552fdbe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:26.718683 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:26.727901 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00edc3f6-bf56-4639-a096-39e13d610461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:26.729622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00edc3f6-bf56-4639-a096-39e13d610461 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:26.729622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00edc3f6-bf56-4639-a096-39e13d610461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:26.729622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00edc3f6-bf56-4639-a096-39e13d610461 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:26.729919 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00edc3f6-bf56-4639-a096-39e13d610461 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:26.731901 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00edc3f6-bf56-4639-a096-39e13d610461 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:26.757246 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1115/2222] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:42:26 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:42:26.758284 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00edc3f6-bf56-4639-a096-39e13d610461 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:26.762260 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b5d22-0ea1-4162-a6ef-5ce1552fdbe1 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:26.767205 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998600f6-5657-4179-930c-0dd8a8718aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:26.768406 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998600f6-5657-4179-930c-0dd8a8718aab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:26.768735 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00edc3f6-bf56-4639-a096-39e13d610461 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:26.768916 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998600f6-5657-4179-930c-0dd8a8718aab admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:26.769084 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998600f6-5657-4179-930c-0dd8a8718aab admin admin] PATH_INFO: `/v3/projects/69d8f8cde4de4e269ec5e2858835b8a5` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:26.770098 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998600f6-5657-4179-930c-0dd8a8718aab admin admin] RBAC: Authorizing `identity:delete_project(project_id=69d8f8cde4de4e269ec5e2858835b8a5)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:26.772045 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998600f6-5657-4179-930c-0dd8a8718aab admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:26.773611 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927bbd60-0dd2-4586-92ac-1701b2f860ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:26.774200 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927bbd60-0dd2-4586-92ac-1701b2f860ac admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:26.774485 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927bbd60-0dd2-4586-92ac-1701b2f860ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:26.774759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927bbd60-0dd2-4586-92ac-1701b2f860ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:26.775447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927bbd60-0dd2-4586-92ac-1701b2f860ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:26.777133 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927bbd60-0dd2-4586-92ac-1701b2f860ac admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:26.787548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1116/2223] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:42:26.791486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927bbd60-0dd2-4586-92ac-1701b2f860ac admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:26.801068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927bbd60-0dd2-4586-92ac-1701b2f860ac admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:26.806045 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9227fac-b347-45c4-bf80-41df164bb806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:26.806684 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9227fac-b347-45c4-bf80-41df164bb806 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:26.806966 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9227fac-b347-45c4-bf80-41df164bb806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:26.807241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9227fac-b347-45c4-bf80-41df164bb806 admin admin] PATH_INFO: `/v3/projects/f245c35fee72467f94bbd1234bd7bdf3/users/93b505e914474c91b02f03025036265c/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:26.822575 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-998600f6-5657-4179-930c-0dd8a8718aab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69d8f8cde4de4e269ec5e2858835b8a5', 'request_id': 'req-998600f6-5657-4179-930c-0dd8a8718aab'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:26.822860 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9227fac-b347-45c4-bf80-41df164bb806 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f245c35fee72467f94bbd1234bd7bdf3, user_id=93b505e914474c91b02f03025036265c, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:26.826690 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9227fac-b347-45c4-bf80-41df164bb806 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:26.828826 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-998600f6-5657-4179-930c-0dd8a8718aab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69d8f8cde4de4e269ec5e2858835b8a5', 'request_id': 'req-998600f6-5657-4179-930c-0dd8a8718aab'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:26.837316 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-998600f6-5657-4179-930c-0dd8a8718aab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69d8f8cde4de4e269ec5e2858835b8a5', 'request_id': 'req-998600f6-5657-4179-930c-0dd8a8718aab'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:26.843644 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1117/2224] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:26 2022] PUT /identity/v3/projects/f245c35fee72467f94bbd1234bd7bdf3/users/93b505e914474c91b02f03025036265c/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:26.844716 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9227fac-b347-45c4-bf80-41df164bb806 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:26.847772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-998600f6-5657-4179-930c-0dd8a8718aab admin admin] The token cache is being invalidate because project 69d8f8cde4de4e269ec5e2858835b8a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:42:26.848063 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-998600f6-5657-4179-930c-0dd8a8718aab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:26.848714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-998600f6-5657-4179-930c-0dd8a8718aab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:26.850274 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1108/2225] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:42:26 2022] DELETE /identity/v3/projects/69d8f8cde4de4e269ec5e2858835b8a5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:26.854232 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998600f6-5657-4179-930c-0dd8a8718aab admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:26.856155 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-998600f6-5657-4179-930c-0dd8a8718aab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:26.905617 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9227fac-b347-45c4-bf80-41df164bb806 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:26.911189 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576a090-54c9-47c8-b1cb-aafd3f06d755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:26.911784 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576a090-54c9-47c8-b1cb-aafd3f06d755 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:26.912062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576a090-54c9-47c8-b1cb-aafd3f06d755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:26.912328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576a090-54c9-47c8-b1cb-aafd3f06d755 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:26.912989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576a090-54c9-47c8-b1cb-aafd3f06d755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:26.914581 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576a090-54c9-47c8-b1cb-aafd3f06d755 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:26.923411 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998600f6-5657-4179-930c-0dd8a8718aab admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:26.925895 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1118/2226] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:26.928976 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c4f02-ba7e-4d4b-b317-c518ee485e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:26.929873 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576a090-54c9-47c8-b1cb-aafd3f06d755 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:26.930925 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c4f02-ba7e-4d4b-b317-c518ee485e4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:26.931335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c4f02-ba7e-4d4b-b317-c518ee485e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:26.931577 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-5576a090-54c9-47c8-b1cb-aafd3f06d755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:26.931680 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c4f02-ba7e-4d4b-b317-c518ee485e4a admin admin] PATH_INFO: `/v3/users/a0f546b4cdd74c459f77b22f183d8725` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:26.932774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c4f02-ba7e-4d4b-b317-c518ee485e4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0f546b4cdd74c459f77b22f183d8725)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:26.934731 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c4f02-ba7e-4d4b-b317-c518ee485e4a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:26.946896 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576a090-54c9-47c8-b1cb-aafd3f06d755 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:26.952690 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8a808-4547-49a6-b2e5-f1e1410ae3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:26.953433 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8a808-4547-49a6-b2e5-f1e1410ae3c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:26.953718 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8a808-4547-49a6-b2e5-f1e1410ae3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:26.954039 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8a808-4547-49a6-b2e5-f1e1410ae3c3 admin admin] PATH_INFO: `/v3/projects/f245c35fee72467f94bbd1234bd7bdf3/users/93b505e914474c91b02f03025036265c/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:26.957398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c8a808-4547-49a6-b2e5-f1e1410ae3c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f245c35fee72467f94bbd1234bd7bdf3, user_id=93b505e914474c91b02f03025036265c, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:26.960694 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c8a808-4547-49a6-b2e5-f1e1410ae3c3 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:26.963589 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:26.975549 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1119/2227] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:26 2022] PUT /identity/v3/projects/f245c35fee72467f94bbd1234bd7bdf3/users/93b505e914474c91b02f03025036265c/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:26.986275 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-509c4f02-ba7e-4d4b-b317-c518ee485e4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0f546b4cdd74c459f77b22f183d8725'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:26.987564 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea26e6d-a12d-43dc-af6f-55f2e33270ca None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:26.987834 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea26e6d-a12d-43dc-af6f-55f2e33270ca None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:26.987939 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-509c4f02-ba7e-4d4b-b317-c518ee485e4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0f546b4cdd74c459f77b22f183d8725'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:26.988095 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea26e6d-a12d-43dc-af6f-55f2e33270ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:26.993592 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:26.995722 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-509c4f02-ba7e-4d4b-b317-c518ee485e4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0f546b4cdd74c459f77b22f183d8725'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:27.004952 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-509c4f02-ba7e-4d4b-b317-c518ee485e4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0f546b4cdd74c459f77b22f183d8725'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:27.006608 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-509c4f02-ba7e-4d4b-b317-c518ee485e4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0f546b4cdd74c459f77b22f183d8725'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:27.019180 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-bea26e6d-a12d-43dc-af6f-55f2e33270ca None None] MFA Rules not processed for user `93b505e914474c91b02f03025036265c`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:27.020498 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-bea26e6d-a12d-43dc-af6f-55f2e33270ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:27.024531 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1109/2228] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:42:26 2022] DELETE /identity/v3/users/a0f546b4cdd74c459f77b22f183d8725 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:27.091887 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1120/2229] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:42:27.103549 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c4f02-ba7e-4d4b-b317-c518ee485e4a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:27.105089 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-509c4f02-ba7e-4d4b-b317-c518ee485e4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:27.168076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c4f02-ba7e-4d4b-b317-c518ee485e4a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:27.173476 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34317a8-4a72-4689-a3ab-dc6a9bf54422 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:27.174006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34317a8-4a72-4689-a3ab-dc6a9bf54422 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:27.174526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34317a8-4a72-4689-a3ab-dc6a9bf54422 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:27.174812 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34317a8-4a72-4689-a3ab-dc6a9bf54422 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:27.240646 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34317a8-4a72-4689-a3ab-dc6a9bf54422 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:27.242115 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34317a8-4a72-4689-a3ab-dc6a9bf54422 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:27.251992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1110/2230] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:42:27 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:42:27.318204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea26e6d-a12d-43dc-af6f-55f2e33270ca None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:27.320018 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-bea26e6d-a12d-43dc-af6f-55f2e33270ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:27.402586 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea26e6d-a12d-43dc-af6f-55f2e33270ca None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:27.465436 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36135420-8e51-4294-9a5c-b448c8dc66a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:27.466392 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36135420-8e51-4294-9a5c-b448c8dc66a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:27.466392 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36135420-8e51-4294-9a5c-b448c8dc66a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:27.466671 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36135420-8e51-4294-9a5c-b448c8dc66a2 admin admin] PATH_INFO: `/v3/projects/72ccc6add928467da2c36f2bbedc0b52` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:27.467800 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36135420-8e51-4294-9a5c-b448c8dc66a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72ccc6add928467da2c36f2bbedc0b52)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:27.469856 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36135420-8e51-4294-9a5c-b448c8dc66a2 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:27.509709 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:INSERT=1,keystone:SELECT=97,keystone:DELETE=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:27.527943 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-36135420-8e51-4294-9a5c-b448c8dc66a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72ccc6add928467da2c36f2bbedc0b52', 'request_id': 'req-36135420-8e51-4294-9a5c-b448c8dc66a2'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:27.535108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-36135420-8e51-4294-9a5c-b448c8dc66a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72ccc6add928467da2c36f2bbedc0b52', 'request_id': 'req-36135420-8e51-4294-9a5c-b448c8dc66a2'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:27.544078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-36135420-8e51-4294-9a5c-b448c8dc66a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72ccc6add928467da2c36f2bbedc0b52', 'request_id': 'req-36135420-8e51-4294-9a5c-b448c8dc66a2'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:27.556279 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-36135420-8e51-4294-9a5c-b448c8dc66a2 admin admin] The token cache is being invalidate because project 72ccc6add928467da2c36f2bbedc0b52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:42:27.556573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-36135420-8e51-4294-9a5c-b448c8dc66a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:27.557237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-36135420-8e51-4294-9a5c-b448c8dc66a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:27.558733 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1121/2231] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:42:27 2022] DELETE /identity/v3/projects/72ccc6add928467da2c36f2bbedc0b52 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:27.565089 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79526bac-380a-49cf-a4d9-222450011d76 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:27.565372 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79526bac-380a-49cf-a4d9-222450011d76 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:27.565639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79526bac-380a-49cf-a4d9-222450011d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:27.586995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-79526bac-380a-49cf-a4d9-222450011d76 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:27.588188 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-79526bac-380a-49cf-a4d9-222450011d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:27.650097 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1111/2232] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:27.658762 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aab992-5f74-466c-9362-2cbd5cf311c2 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:27.659114 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aab992-5f74-466c-9362-2cbd5cf311c2 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:27.659389 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aab992-5f74-466c-9362-2cbd5cf311c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:27.681740 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-17aab992-5f74-466c-9362-2cbd5cf311c2 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:27.683273 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-17aab992-5f74-466c-9362-2cbd5cf311c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:27.691907 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79526bac-380a-49cf-a4d9-222450011d76 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:27.692442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1122/2233] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:27.694057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-79526bac-380a-49cf-a4d9-222450011d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:27.696345 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aab992-5f74-466c-9362-2cbd5cf311c2 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:27.705483 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aab992-5f74-466c-9362-2cbd5cf311c2 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:27.706081 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79526bac-380a-49cf-a4d9-222450011d76 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:27.710111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3d009-fa4a-40ab-995a-ef7b74d2a4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:27.710616 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102b17d-87b4-4298-a6a8-15cc946c61dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:27.710751 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3d009-fa4a-40ab-995a-ef7b74d2a4ba admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:27.711038 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3d009-fa4a-40ab-995a-ef7b74d2a4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:27.711234 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102b17d-87b4-4298-a6a8-15cc946c61dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:27.711341 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3d009-fa4a-40ab-995a-ef7b74d2a4ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:27.711514 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102b17d-87b4-4298-a6a8-15cc946c61dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:27.711786 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102b17d-87b4-4298-a6a8-15cc946c61dd admin admin] PATH_INFO: `/v3/users/3b1ef7d93a474cbf96de059d535a9cdf` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:27.711948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3d009-fa4a-40ab-995a-ef7b74d2a4ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:42:27.712253 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3d009-fa4a-40ab-995a-ef7b74d2a4ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:27.712930 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102b17d-87b4-4298-a6a8-15cc946c61dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b1ef7d93a474cbf96de059d535a9cdf)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:27.713891 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3d009-fa4a-40ab-995a-ef7b74d2a4ba admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:27.714870 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102b17d-87b4-4298-a6a8-15cc946c61dd admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:27.726929 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1123/2234] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:42:27.731143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3d009-fa4a-40ab-995a-ef7b74d2a4ba admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:27.732566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:27.741609 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3d009-fa4a-40ab-995a-ef7b74d2a4ba admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:27.747800 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19247c4d-01ba-49e4-bb89-0ad95762ee69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:27.748439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19247c4d-01ba-49e4-bb89-0ad95762ee69 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:27.748722 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19247c4d-01ba-49e4-bb89-0ad95762ee69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:27.749482 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19247c4d-01ba-49e4-bb89-0ad95762ee69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:27.750134 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19247c4d-01ba-49e4-bb89-0ad95762ee69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:27.753089 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19247c4d-01ba-49e4-bb89-0ad95762ee69 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:27.772110 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1102b17d-87b4-4298-a6a8-15cc946c61dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b1ef7d93a474cbf96de059d535a9cdf'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:27.772884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1102b17d-87b4-4298-a6a8-15cc946c61dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b1ef7d93a474cbf96de059d535a9cdf'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:27.780657 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1102b17d-87b4-4298-a6a8-15cc946c61dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b1ef7d93a474cbf96de059d535a9cdf'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:27.789600 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1124/2235] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:42:27 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:42:27.789986 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1102b17d-87b4-4298-a6a8-15cc946c61dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b1ef7d93a474cbf96de059d535a9cdf'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:27.790817 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-1102b17d-87b4-4298-a6a8-15cc946c61dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b1ef7d93a474cbf96de059d535a9cdf'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:27.796572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19247c4d-01ba-49e4-bb89-0ad95762ee69 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:27.799210 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-19247c4d-01ba-49e4-bb89-0ad95762ee69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:27.814143 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1112/2236] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:42:27 2022] DELETE /identity/v3/users/3b1ef7d93a474cbf96de059d535a9cdf => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:27.819013 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fc955-2b03-42fb-80e8-74a72073cb62 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:27.819532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fc955-2b03-42fb-80e8-74a72073cb62 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:27.819701 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fc955-2b03-42fb-80e8-74a72073cb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:27.840674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-ac6fc955-2b03-42fb-80e8-74a72073cb62 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:27.842061 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ac6fc955-2b03-42fb-80e8-74a72073cb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:27.902017 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19247c4d-01ba-49e4-bb89-0ad95762ee69 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:27.950694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1113/2237] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:42:27.960578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fc955-2b03-42fb-80e8-74a72073cb62 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:27.961981 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ac6fc955-2b03-42fb-80e8-74a72073cb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:27.972163 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108163a-1485-4243-b258-2504fba455b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:27.973565 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108163a-1485-4243-b258-2504fba455b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:27.973845 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108163a-1485-4243-b258-2504fba455b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:27.974116 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108163a-1485-4243-b258-2504fba455b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:27.974751 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108163a-1485-4243-b258-2504fba455b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:27.976679 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108163a-1485-4243-b258-2504fba455b7 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:28.002756 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1125/2238] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:42:27 2022] POST /identity/v3/users => generated 287 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:42:28.006642 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108163a-1485-4243-b258-2504fba455b7 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:28.018122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108163a-1485-4243-b258-2504fba455b7 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:28.023237 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7797a-8b4a-41f2-8aa7-adbd6e97cea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:28.023926 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7797a-8b4a-41f2-8aa7-adbd6e97cea2 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:28.024212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7797a-8b4a-41f2-8aa7-adbd6e97cea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:28.024621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7797a-8b4a-41f2-8aa7-adbd6e97cea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:28.025314 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7797a-8b4a-41f2-8aa7-adbd6e97cea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:28.027052 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7797a-8b4a-41f2-8aa7-adbd6e97cea2 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:28.030606 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fc955-2b03-42fb-80e8-74a72073cb62 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:28.033898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:28.035542 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bee59a-a511-4a03-8b92-8e8dc959a867 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:28.036657 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bee59a-a511-4a03-8b92-8e8dc959a867 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:28.037154 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bee59a-a511-4a03-8b92-8e8dc959a867 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:28.038016 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bee59a-a511-4a03-8b92-8e8dc959a867 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:28.043466 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1126/2239] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:42:28.044437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7797a-8b4a-41f2-8aa7-adbd6e97cea2 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:28.053560 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bee59a-a511-4a03-8b92-8e8dc959a867 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:28.055047 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bee59a-a511-4a03-8b92-8e8dc959a867 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:28.056634 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7797a-8b4a-41f2-8aa7-adbd6e97cea2 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:28.062303 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4a2a9-18c9-4504-9168-99f916759a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:28.062912 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4a2a9-18c9-4504-9168-99f916759a66 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:28.063269 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4a2a9-18c9-4504-9168-99f916759a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:28.063457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4a2a9-18c9-4504-9168-99f916759a66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:28.064091 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d4a2a9-18c9-4504-9168-99f916759a66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:28.065951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d4a2a9-18c9-4504-9168-99f916759a66 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:28.066563 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1114/2240] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:42:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:42:28.077962 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1127/2241] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:28.083144 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bee59a-a511-4a03-8b92-8e8dc959a867 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:28.094227 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bee59a-a511-4a03-8b92-8e8dc959a867 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:28.099673 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c3a0b-8dea-42e4-bb0a-335e8cdc2359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:28.100349 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c3a0b-8dea-42e4-bb0a-335e8cdc2359 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:28.100635 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c3a0b-8dea-42e4-bb0a-335e8cdc2359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:28.100963 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c3a0b-8dea-42e4-bb0a-335e8cdc2359 admin admin] PATH_INFO: `/v3/projects/e0fa16780b0b460f94a3ead14e186b01/users/d31aad3e6f094f68aa01442e15de2f9f/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:28.121207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c3a0b-8dea-42e4-bb0a-335e8cdc2359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0fa16780b0b460f94a3ead14e186b01, user_id=d31aad3e6f094f68aa01442e15de2f9f, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:28.124436 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c3a0b-8dea-42e4-bb0a-335e8cdc2359 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:28.138709 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1115/2242] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:28 2022] PUT /identity/v3/projects/e0fa16780b0b460f94a3ead14e186b01/users/d31aad3e6f094f68aa01442e15de2f9f/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:28.150119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf27f84-533b-455d-95ca-7a6906f0fd7e None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:28.150461 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf27f84-533b-455d-95ca-7a6906f0fd7e None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:28.150683 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf27f84-533b-455d-95ca-7a6906f0fd7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:28.182860 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-4bf27f84-533b-455d-95ca-7a6906f0fd7e None None] MFA Rules not processed for user `d31aad3e6f094f68aa01442e15de2f9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:28.184079 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-4bf27f84-533b-455d-95ca-7a6906f0fd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:28.259761 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1128/2243] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:28.264300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c3a0b-8dea-42e4-bb0a-335e8cdc2359 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:28.278887 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:28.332315 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c3a0b-8dea-42e4-bb0a-335e8cdc2359 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:28.338518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223894c-29a8-4ee4-8b84-ad51a20cf78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:28.339523 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223894c-29a8-4ee4-8b84-ad51a20cf78a admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:28.340041 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223894c-29a8-4ee4-8b84-ad51a20cf78a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:28.340892 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223894c-29a8-4ee4-8b84-ad51a20cf78a admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:28.341786 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4223894c-29a8-4ee4-8b84-ad51a20cf78a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:28.346613 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4223894c-29a8-4ee4-8b84-ad51a20cf78a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:28.357613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf27f84-533b-455d-95ca-7a6906f0fd7e None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:28.359257 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-4bf27f84-533b-455d-95ca-7a6906f0fd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:28.380037 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf27f84-533b-455d-95ca-7a6906f0fd7e None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:28.386288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a123c-e5d0-4986-a8ac-b790aae02733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:28.386870 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a123c-e5d0-4986-a8ac-b790aae02733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:28.387151 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a123c-e5d0-4986-a8ac-b790aae02733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:28.387428 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a123c-e5d0-4986-a8ac-b790aae02733 admin admin] PATH_INFO: `/v3/projects/27a8f2a584c4414b89c877dd143c6207` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:28.388662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a123c-e5d0-4986-a8ac-b790aae02733 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27a8f2a584c4414b89c877dd143c6207)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:28.390951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a123c-e5d0-4986-a8ac-b790aae02733 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:28.391595 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1116/2244] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:42:28 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:42:28.395459 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223894c-29a8-4ee4-8b84-ad51a20cf78a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:28.452687 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-b31a123c-e5d0-4986-a8ac-b790aae02733 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27a8f2a584c4414b89c877dd143c6207', 'request_id': 'req-b31a123c-e5d0-4986-a8ac-b790aae02733'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:28.460503 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-b31a123c-e5d0-4986-a8ac-b790aae02733 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27a8f2a584c4414b89c877dd143c6207', 'request_id': 'req-b31a123c-e5d0-4986-a8ac-b790aae02733'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:28.469264 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223894c-29a8-4ee4-8b84-ad51a20cf78a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:28.470068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-b31a123c-e5d0-4986-a8ac-b790aae02733 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27a8f2a584c4414b89c877dd143c6207', 'request_id': 'req-b31a123c-e5d0-4986-a8ac-b790aae02733'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:28.487849 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-b31a123c-e5d0-4986-a8ac-b790aae02733 admin admin] The token cache is being invalidate because project 27a8f2a584c4414b89c877dd143c6207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:42:28.488539 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-b31a123c-e5d0-4986-a8ac-b790aae02733 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:28.489247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-b31a123c-e5d0-4986-a8ac-b790aae02733 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:28.490917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1129/2245] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:42:28 2022] DELETE /identity/v3/projects/27a8f2a584c4414b89c877dd143c6207 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:28.495378 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a123c-e5d0-4986-a8ac-b790aae02733 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:28.497283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b31a123c-e5d0-4986-a8ac-b790aae02733 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:28.538749 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba74a1-d759-4741-a578-c7ca828e18b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:28.539355 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba74a1-d759-4741-a578-c7ca828e18b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:28.539634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba74a1-d759-4741-a578-c7ca828e18b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:28.540529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba74a1-d759-4741-a578-c7ca828e18b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:28.541230 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba74a1-d759-4741-a578-c7ca828e18b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:28.543121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba74a1-d759-4741-a578-c7ca828e18b0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:28.568525 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1117/2246] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:42:28 2022] POST /identity/v3/users => generated 294 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:42:28.569922 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba74a1-d759-4741-a578-c7ca828e18b0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:28.571255 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a123c-e5d0-4986-a8ac-b790aae02733 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:28.571556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-dbba74a1-d759-4741-a578-c7ca828e18b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:28.576128 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d6faf-3a00-4305-98a4-53bf73a05105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:28.576758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d6faf-3a00-4305-98a4-53bf73a05105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:28.577074 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d6faf-3a00-4305-98a4-53bf73a05105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:28.577354 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d6faf-3a00-4305-98a4-53bf73a05105 admin admin] PATH_INFO: `/v3/users/93b505e914474c91b02f03025036265c` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:28.578615 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1d6faf-3a00-4305-98a4-53bf73a05105 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93b505e914474c91b02f03025036265c)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:28.580476 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1d6faf-3a00-4305-98a4-53bf73a05105 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:28.582910 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:28.583590 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba74a1-d759-4741-a578-c7ca828e18b0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:28.588820 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f21a71-0051-4ac2-97c4-663084c53176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:28.589442 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f21a71-0051-4ac2-97c4-663084c53176 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:28.589724 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f21a71-0051-4ac2-97c4-663084c53176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:28.590006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f21a71-0051-4ac2-97c4-663084c53176 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:28.590641 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f21a71-0051-4ac2-97c4-663084c53176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:28.592233 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f21a71-0051-4ac2-97c4-663084c53176 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:28.602676 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1118/2247] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:42:28.606992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f21a71-0051-4ac2-97c4-663084c53176 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:28.618641 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f21a71-0051-4ac2-97c4-663084c53176 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:28.623946 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8853b-150f-43a2-92df-b7d66df45153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:28.624590 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8853b-150f-43a2-92df-b7d66df45153 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:28.624935 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8853b-150f-43a2-92df-b7d66df45153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:28.625235 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8853b-150f-43a2-92df-b7d66df45153 admin admin] PATH_INFO: `/v3/projects/8262b5cdea114008aea59b1f9a8f7e54/users/3650702d6adb4cf39b37cecd7de48dd5/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:28.631320 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0e1d6faf-3a00-4305-98a4-53bf73a05105 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93b505e914474c91b02f03025036265c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:28.631986 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0e1d6faf-3a00-4305-98a4-53bf73a05105 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93b505e914474c91b02f03025036265c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:28.640418 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0e1d6faf-3a00-4305-98a4-53bf73a05105 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93b505e914474c91b02f03025036265c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:28.641813 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0e1d6faf-3a00-4305-98a4-53bf73a05105 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93b505e914474c91b02f03025036265c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:28.642922 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b8853b-150f-43a2-92df-b7d66df45153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8262b5cdea114008aea59b1f9a8f7e54, user_id=3650702d6adb4cf39b37cecd7de48dd5, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:28.645607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b8853b-150f-43a2-92df-b7d66df45153 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:28.649862 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0e1d6faf-3a00-4305-98a4-53bf73a05105 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93b505e914474c91b02f03025036265c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:28.656140 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:28.664234 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1119/2248] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:28 2022] PUT /identity/v3/projects/8262b5cdea114008aea59b1f9a8f7e54/users/3650702d6adb4cf39b37cecd7de48dd5/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:28.665258 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8853b-150f-43a2-92df-b7d66df45153 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:28.667028 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d8b8853b-150f-43a2-92df-b7d66df45153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:28.669740 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1130/2249] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:42:28 2022] DELETE /identity/v3/users/93b505e914474c91b02f03025036265c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:28.745512 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8853b-150f-43a2-92df-b7d66df45153 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:28.803686 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc387b16-3c7d-41f2-9df4-eefa413edee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:28.804301 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc387b16-3c7d-41f2-9df4-eefa413edee9 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:28.804581 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc387b16-3c7d-41f2-9df4-eefa413edee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:28.804855 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc387b16-3c7d-41f2-9df4-eefa413edee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:28.805529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc387b16-3c7d-41f2-9df4-eefa413edee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:28.807200 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc387b16-3c7d-41f2-9df4-eefa413edee9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:28.820302 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1120/2250] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:42:28.821821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d6faf-3a00-4305-98a4-53bf73a05105 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:28.832526 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d6faf-3a00-4305-98a4-53bf73a05105 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:28.837095 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40147be-6fb9-47f1-81d6-decde5bb8bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:28.837771 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40147be-6fb9-47f1-81d6-decde5bb8bcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:28.838032 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40147be-6fb9-47f1-81d6-decde5bb8bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:28.838305 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40147be-6fb9-47f1-81d6-decde5bb8bcb admin admin] PATH_INFO: `/v3/projects/8262b5cdea114008aea59b1f9a8f7e54/users/3650702d6adb4cf39b37cecd7de48dd5/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:28.840197 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40147be-6fb9-47f1-81d6-decde5bb8bcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8262b5cdea114008aea59b1f9a8f7e54, user_id=3650702d6adb4cf39b37cecd7de48dd5, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:28.842858 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40147be-6fb9-47f1-81d6-decde5bb8bcb admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:28.853854 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1131/2251] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:28 2022] PUT /identity/v3/projects/8262b5cdea114008aea59b1f9a8f7e54/users/3650702d6adb4cf39b37cecd7de48dd5/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:28.865555 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66ca58-9d1c-4c84-9391-1d4eb08bb4ab None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:28.865849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66ca58-9d1c-4c84-9391-1d4eb08bb4ab None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:28.866130 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66ca58-9d1c-4c84-9391-1d4eb08bb4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:28.895747 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5b66ca58-9d1c-4c84-9391-1d4eb08bb4ab None None] MFA Rules not processed for user `3650702d6adb4cf39b37cecd7de48dd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:28.897004 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5b66ca58-9d1c-4c84-9391-1d4eb08bb4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:28.957389 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40147be-6fb9-47f1-81d6-decde5bb8bcb admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:28.959517 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-f40147be-6fb9-47f1-81d6-decde5bb8bcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:28.981647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1121/2252] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:28.994892 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b66ca58-9d1c-4c84-9391-1d4eb08bb4ab None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:28.996358 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5b66ca58-9d1c-4c84-9391-1d4eb08bb4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:29.032002 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:29.056930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40147be-6fb9-47f1-81d6-decde5bb8bcb admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:29.061133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b66ca58-9d1c-4c84-9391-1d4eb08bb4ab None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:29.061580 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295eb4d9-b789-42ef-9591-8298a3adbfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:29.062214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295eb4d9-b789-42ef-9591-8298a3adbfeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:29.062492 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295eb4d9-b789-42ef-9591-8298a3adbfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:29.062763 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295eb4d9-b789-42ef-9591-8298a3adbfeb admin admin] PATH_INFO: `/v3/projects/f245c35fee72467f94bbd1234bd7bdf3` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:29.065039 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295eb4d9-b789-42ef-9591-8298a3adbfeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f245c35fee72467f94bbd1234bd7bdf3)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:29.066192 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91780a0-df67-4fda-9225-1dbb9fd88b58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:29.066875 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91780a0-df67-4fda-9225-1dbb9fd88b58 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:29.066992 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295eb4d9-b789-42ef-9591-8298a3adbfeb admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:29.067243 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91780a0-df67-4fda-9225-1dbb9fd88b58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:29.067592 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91780a0-df67-4fda-9225-1dbb9fd88b58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:29.069612 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b91780a0-df67-4fda-9225-1dbb9fd88b58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:29.130713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-295eb4d9-b789-42ef-9591-8298a3adbfeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f245c35fee72467f94bbd1234bd7bdf3', 'request_id': 'req-295eb4d9-b789-42ef-9591-8298a3adbfeb'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:29.134691 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91780a0-df67-4fda-9225-1dbb9fd88b58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:29.136127 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91780a0-df67-4fda-9225-1dbb9fd88b58 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:29.137083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-295eb4d9-b789-42ef-9591-8298a3adbfeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f245c35fee72467f94bbd1234bd7bdf3', 'request_id': 'req-295eb4d9-b789-42ef-9591-8298a3adbfeb'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:29.145606 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-295eb4d9-b789-42ef-9591-8298a3adbfeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f245c35fee72467f94bbd1234bd7bdf3', 'request_id': 'req-295eb4d9-b789-42ef-9591-8298a3adbfeb'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:29.147087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1122/2253] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:42:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:42:29.152348 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:29.157359 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-295eb4d9-b789-42ef-9591-8298a3adbfeb admin admin] The token cache is being invalidate because project f245c35fee72467f94bbd1234bd7bdf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:42:29.157833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-295eb4d9-b789-42ef-9591-8298a3adbfeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:29.159222 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-295eb4d9-b789-42ef-9591-8298a3adbfeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:29.161078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1132/2254] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:42:28 2022] DELETE /identity/v3/projects/f245c35fee72467f94bbd1234bd7bdf3 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:29.166009 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b522741-4d6b-4d92-ae8b-c209c1a4d6c5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:29.166301 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b522741-4d6b-4d92-ae8b-c209c1a4d6c5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:29.166576 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b522741-4d6b-4d92-ae8b-c209c1a4d6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:29.186431 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-5b522741-4d6b-4d92-ae8b-c209c1a4d6c5 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:29.187772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5b522741-4d6b-4d92-ae8b-c209c1a4d6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:29.252890 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1123/2255] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:29.260488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664167f-c53e-4474-a7ce-0dfa9ab8618b None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:29.260773 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664167f-c53e-4474-a7ce-0dfa9ab8618b None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:29.261077 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664167f-c53e-4474-a7ce-0dfa9ab8618b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:29.283002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-8664167f-c53e-4474-a7ce-0dfa9ab8618b None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:29.284362 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-8664167f-c53e-4474-a7ce-0dfa9ab8618b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:29.293113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1133/2256] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:29.297482 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b522741-4d6b-4d92-ae8b-c209c1a4d6c5 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:29.308176 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b522741-4d6b-4d92-ae8b-c209c1a4d6c5 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:29.312973 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873cc221-14ab-4e07-ad5b-0e390452dcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:29.313580 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873cc221-14ab-4e07-ad5b-0e390452dcb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:29.313862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873cc221-14ab-4e07-ad5b-0e390452dcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:29.314138 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873cc221-14ab-4e07-ad5b-0e390452dcb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:29.314886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873cc221-14ab-4e07-ad5b-0e390452dcb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:42:29.315195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873cc221-14ab-4e07-ad5b-0e390452dcb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:29.316853 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873cc221-14ab-4e07-ad5b-0e390452dcb9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:29.332983 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1124/2257] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:42:29.334084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664167f-c53e-4474-a7ce-0dfa9ab8618b None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:29.344086 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664167f-c53e-4474-a7ce-0dfa9ab8618b None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:29.350030 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771aba3-bae5-4fd6-ae1d-7bd9c1b8b4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:29.350628 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771aba3-bae5-4fd6-ae1d-7bd9c1b8b4c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:29.350906 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771aba3-bae5-4fd6-ae1d-7bd9c1b8b4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:29.351183 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771aba3-bae5-4fd6-ae1d-7bd9c1b8b4c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:29.351905 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771aba3-bae5-4fd6-ae1d-7bd9c1b8b4c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:29.354048 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771aba3-bae5-4fd6-ae1d-7bd9c1b8b4c5 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:29.386921 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1134/2258] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:42:29 2022] POST /identity/v3/projects => generated 355 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:42:29.388297 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873cc221-14ab-4e07-ad5b-0e390452dcb9 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:29.399415 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873cc221-14ab-4e07-ad5b-0e390452dcb9 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:29.468772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b26155-8cd6-47a2-b1f0-0f01b01e6420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:29.469369 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b26155-8cd6-47a2-b1f0-0f01b01e6420 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:29.469650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b26155-8cd6-47a2-b1f0-0f01b01e6420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:29.469958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b26155-8cd6-47a2-b1f0-0f01b01e6420 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:29.470745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b26155-8cd6-47a2-b1f0-0f01b01e6420 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:29.473029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b26155-8cd6-47a2-b1f0-0f01b01e6420 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:29.491905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:29.498031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1125/2259] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:42:29 2022] POST /identity/v3/users => generated 278 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:42:29.502187 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771aba3-bae5-4fd6-ae1d-7bd9c1b8b4c5 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:29.513826 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771aba3-bae5-4fd6-ae1d-7bd9c1b8b4c5 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:29.519330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4729c-8bc7-49d3-902e-a3418e547ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:29.519974 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4729c-8bc7-49d3-902e-a3418e547ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:29.520284 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4729c-8bc7-49d3-902e-a3418e547ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:29.520600 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4729c-8bc7-49d3-902e-a3418e547ea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:29.521487 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b4729c-8bc7-49d3-902e-a3418e547ea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:29.523165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b4729c-8bc7-49d3-902e-a3418e547ea9 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:29.534654 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1135/2260] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:29.538585 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b26155-8cd6-47a2-b1f0-0f01b01e6420 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:29.548501 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b26155-8cd6-47a2-b1f0-0f01b01e6420 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:29.553386 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56667c47-3074-4996-b14d-bb6f01cc007d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:29.554029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56667c47-3074-4996-b14d-bb6f01cc007d admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:29.554317 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56667c47-3074-4996-b14d-bb6f01cc007d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:29.554590 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56667c47-3074-4996-b14d-bb6f01cc007d admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:29.555239 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56667c47-3074-4996-b14d-bb6f01cc007d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:29.556840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56667c47-3074-4996-b14d-bb6f01cc007d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:29.567616 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1126/2261] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:29.571778 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4729c-8bc7-49d3-902e-a3418e547ea9 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:29.582384 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4729c-8bc7-49d3-902e-a3418e547ea9 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:29.587247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb68dd-4f69-4319-8cc9-e0f8d771fc05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:29.587922 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb68dd-4f69-4319-8cc9-e0f8d771fc05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:29.588205 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb68dd-4f69-4319-8cc9-e0f8d771fc05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:29.588484 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb68dd-4f69-4319-8cc9-e0f8d771fc05 admin admin] PATH_INFO: `/v3/projects/e7895f3e04f344088f59ea5578bb27b0/users/c3f016db213248a687b74bfa6af27e45/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:29.603399 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb68dd-4f69-4319-8cc9-e0f8d771fc05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7895f3e04f344088f59ea5578bb27b0, user_id=c3f016db213248a687b74bfa6af27e45, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:29.606119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb68dd-4f69-4319-8cc9-e0f8d771fc05 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:29.620974 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1136/2262] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:29 2022] PUT /identity/v3/projects/e7895f3e04f344088f59ea5578bb27b0/users/c3f016db213248a687b74bfa6af27e45/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:29.633836 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff1261-738b-46a8-abd2-984c65a723b0 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:29.634120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff1261-738b-46a8-abd2-984c65a723b0 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:29.634393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff1261-738b-46a8-abd2-984c65a723b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:29.662077 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-1eff1261-738b-46a8-abd2-984c65a723b0 None None] MFA Rules not processed for user `c3f016db213248a687b74bfa6af27e45`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:29.663269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1eff1261-738b-46a8-abd2-984c65a723b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:29.727812 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1127/2263] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:29 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:29.732252 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb68dd-4f69-4319-8cc9-e0f8d771fc05 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:29.794575 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb68dd-4f69-4319-8cc9-e0f8d771fc05 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:29.799214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e487128-e8c9-4ac5-bb90-bfba3bdb5fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:29.799805 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e487128-e8c9-4ac5-bb90-bfba3bdb5fc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:29.800081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e487128-e8c9-4ac5-bb90-bfba3bdb5fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:29.800348 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e487128-e8c9-4ac5-bb90-bfba3bdb5fc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:29.801009 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e487128-e8c9-4ac5-bb90-bfba3bdb5fc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:29.802902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e487128-e8c9-4ac5-bb90-bfba3bdb5fc2 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:29.828719 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1137/2264] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:42:29 2022] POST /identity/v3/projects => generated 355 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:42:29.833412 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff1261-738b-46a8-abd2-984c65a723b0 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:29.844373 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff1261-738b-46a8-abd2-984c65a723b0 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:29.906473 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94728837-939c-4256-a102-6603f1658a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:29.906696 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94728837-939c-4256-a102-6603f1658a28 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:29.906977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94728837-939c-4256-a102-6603f1658a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:29.907255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94728837-939c-4256-a102-6603f1658a28 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:29.907919 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94728837-939c-4256-a102-6603f1658a28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:29.910411 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94728837-939c-4256-a102-6603f1658a28 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:29.935931 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1128/2265] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:42:29 2022] POST /identity/v3/users => generated 284 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:42:29.936942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e487128-e8c9-4ac5-bb90-bfba3bdb5fc2 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:29.948297 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e487128-e8c9-4ac5-bb90-bfba3bdb5fc2 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:29.952928 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fe01b-28db-4998-a292-3642fd7a4b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:29.953532 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fe01b-28db-4998-a292-3642fd7a4b45 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:29.954821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fe01b-28db-4998-a292-3642fd7a4b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:29.955681 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fe01b-28db-4998-a292-3642fd7a4b45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:29.956337 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605fe01b-28db-4998-a292-3642fd7a4b45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:29.957979 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605fe01b-28db-4998-a292-3642fd7a4b45 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:29.968896 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1138/2266] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:29.973726 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94728837-939c-4256-a102-6603f1658a28 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:29.984273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94728837-939c-4256-a102-6603f1658a28 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:29.990849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f1e9b-d488-476e-b780-6effc2d2679e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:29.991636 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f1e9b-d488-476e-b780-6effc2d2679e admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:29.991919 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f1e9b-d488-476e-b780-6effc2d2679e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:29.992218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f1e9b-d488-476e-b780-6effc2d2679e admin admin] PATH_INFO: `/v3/projects/ec9ab8a43bfd43f0979b785d84e4b6b5/users/ef180bbab3884344ab205ab6a4056618/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:30.010044 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f1e9b-d488-476e-b780-6effc2d2679e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec9ab8a43bfd43f0979b785d84e4b6b5, user_id=ef180bbab3884344ab205ab6a4056618, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:30.012906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f1e9b-d488-476e-b780-6effc2d2679e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:30.032097 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1129/2267] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:29 2022] PUT /identity/v3/projects/ec9ab8a43bfd43f0979b785d84e4b6b5/users/ef180bbab3884344ab205ab6a4056618/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:30.033954 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fe01b-28db-4998-a292-3642fd7a4b45 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:30.044303 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fe01b-28db-4998-a292-3642fd7a4b45 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:30.107088 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f63657-8f4d-4436-a657-07032f37c67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:30.107771 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f63657-8f4d-4436-a657-07032f37c67c admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:30.108031 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f63657-8f4d-4436-a657-07032f37c67c admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:30.108307 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f63657-8f4d-4436-a657-07032f37c67c admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:30.109020 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f63657-8f4d-4436-a657-07032f37c67c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:30.110639 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f63657-8f4d-4436-a657-07032f37c67c admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:30.122331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1139/2268] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:30.125613 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f1e9b-d488-476e-b780-6effc2d2679e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:30.138651 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f1e9b-d488-476e-b780-6effc2d2679e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:30.145327 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e658a7-9fe6-4af5-9423-65c8dbde663d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:30.146086 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e658a7-9fe6-4af5-9423-65c8dbde663d admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:30.146714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e658a7-9fe6-4af5-9423-65c8dbde663d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:30.147067 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e658a7-9fe6-4af5-9423-65c8dbde663d admin admin] PATH_INFO: `/v3/projects/ec9ab8a43bfd43f0979b785d84e4b6b5/users/ef180bbab3884344ab205ab6a4056618/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:30.149123 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e658a7-9fe6-4af5-9423-65c8dbde663d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec9ab8a43bfd43f0979b785d84e4b6b5, user_id=ef180bbab3884344ab205ab6a4056618, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:30.152483 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e658a7-9fe6-4af5-9423-65c8dbde663d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:30.170350 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1130/2269] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:30 2022] PUT /identity/v3/projects/ec9ab8a43bfd43f0979b785d84e4b6b5/users/ef180bbab3884344ab205ab6a4056618/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:30.180992 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd62ed-5254-4e49-a0f1-88e1ed9748c8 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:30.181486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd62ed-5254-4e49-a0f1-88e1ed9748c8 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:30.181772 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd62ed-5254-4e49-a0f1-88e1ed9748c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:30.191983 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:30.215930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-85fd62ed-5254-4e49-a0f1-88e1ed9748c8 None None] MFA Rules not processed for user `ef180bbab3884344ab205ab6a4056618`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:30.217184 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-85fd62ed-5254-4e49-a0f1-88e1ed9748c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:30.289754 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1140/2270] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:30 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:30.301868 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e658a7-9fe6-4af5-9423-65c8dbde663d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:30.304390 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c9e658a7-9fe6-4af5-9423-65c8dbde663d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:30.340200 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:30.365608 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e658a7-9fe6-4af5-9423-65c8dbde663d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:30.369558 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82a55f-c175-4c53-8fdb-a94bae308960 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:30.370166 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82a55f-c175-4c53-8fdb-a94bae308960 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:30.370448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82a55f-c175-4c53-8fdb-a94bae308960 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:30.370720 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e82a55f-c175-4c53-8fdb-a94bae308960 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:30.380500 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82a55f-c175-4c53-8fdb-a94bae308960 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:30.381956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e82a55f-c175-4c53-8fdb-a94bae308960 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:30.391289 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1131/2271] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:42:30 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:42:30.465764 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd62ed-5254-4e49-a0f1-88e1ed9748c8 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:30.476196 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd62ed-5254-4e49-a0f1-88e1ed9748c8 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:30.479953 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6653f2e-4a92-463f-97fd-5953d3859dff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:30.480581 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6653f2e-4a92-463f-97fd-5953d3859dff service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:30.481062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6653f2e-4a92-463f-97fd-5953d3859dff service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:30.481338 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6653f2e-4a92-463f-97fd-5953d3859dff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:30.548299 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6653f2e-4a92-463f-97fd-5953d3859dff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:30.550457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6653f2e-4a92-463f-97fd-5953d3859dff service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:30.559810 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1141/2272] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:42:30 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:42:31.729031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82a55f-c175-4c53-8fdb-a94bae308960 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:31.731101 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9e82a55f-c175-4c53-8fdb-a94bae308960 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:31.811226 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e82a55f-c175-4c53-8fdb-a94bae308960 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:31.817865 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daad05d-15fb-4a65-86d4-37af7bc8d8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:31.818568 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daad05d-15fb-4a65-86d4-37af7bc8d8f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:31.818861 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daad05d-15fb-4a65-86d4-37af7bc8d8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:31.819184 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daad05d-15fb-4a65-86d4-37af7bc8d8f9 admin admin] PATH_INFO: `/v3/users/d31aad3e6f094f68aa01442e15de2f9f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:31.820547 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daad05d-15fb-4a65-86d4-37af7bc8d8f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d31aad3e6f094f68aa01442e15de2f9f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:31.822571 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daad05d-15fb-4a65-86d4-37af7bc8d8f9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:31.886893 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8daad05d-15fb-4a65-86d4-37af7bc8d8f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd31aad3e6f094f68aa01442e15de2f9f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:31.887709 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8daad05d-15fb-4a65-86d4-37af7bc8d8f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd31aad3e6f094f68aa01442e15de2f9f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:31.899787 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8daad05d-15fb-4a65-86d4-37af7bc8d8f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd31aad3e6f094f68aa01442e15de2f9f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:31.912980 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8daad05d-15fb-4a65-86d4-37af7bc8d8f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd31aad3e6f094f68aa01442e15de2f9f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:31.913712 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8daad05d-15fb-4a65-86d4-37af7bc8d8f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd31aad3e6f094f68aa01442e15de2f9f'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:31.934434 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1132/2273] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:42:31 2022] DELETE /identity/v3/users/d31aad3e6f094f68aa01442e15de2f9f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:31.938885 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ada5f-dd6c-4cdf-a267-fbb1656ebce5 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:31.939402 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ada5f-dd6c-4cdf-a267-fbb1656ebce5 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:31.939755 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ada5f-dd6c-4cdf-a267-fbb1656ebce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:31.958953 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-bc1ada5f-dd6c-4cdf-a267-fbb1656ebce5 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:31.960250 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-bc1ada5f-dd6c-4cdf-a267-fbb1656ebce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:32.018403 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1142/2274] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:32.028002 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daad05d-15fb-4a65-86d4-37af7bc8d8f9 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:32.029440 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8daad05d-15fb-4a65-86d4-37af7bc8d8f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:32.087093 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daad05d-15fb-4a65-86d4-37af7bc8d8f9 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:32.090854 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8afd0-c36c-4c90-8528-0100470695ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:32.091408 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8afd0-c36c-4c90-8528-0100470695ba service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:32.091686 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8afd0-c36c-4c90-8528-0100470695ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:32.091964 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8afd0-c36c-4c90-8528-0100470695ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:32.098898 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:32.101672 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8afd0-c36c-4c90-8528-0100470695ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:32.104054 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8afd0-c36c-4c90-8528-0100470695ba service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:32.115302 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1133/2275] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:42:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:42:32.377419 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ada5f-dd6c-4cdf-a267-fbb1656ebce5 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:32.379215 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-bc1ada5f-dd6c-4cdf-a267-fbb1656ebce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:32.392310 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ada5f-dd6c-4cdf-a267-fbb1656ebce5 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:32.397123 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7a50f-397b-4559-a28d-545939afea46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:32.397767 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7a50f-397b-4559-a28d-545939afea46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:32.398050 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7a50f-397b-4559-a28d-545939afea46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:32.398328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7a50f-397b-4559-a28d-545939afea46 admin admin] PATH_INFO: `/v3/projects/e0fa16780b0b460f94a3ead14e186b01` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:32.406381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7a50f-397b-4559-a28d-545939afea46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0fa16780b0b460f94a3ead14e186b01)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:32.408369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7a50f-397b-4559-a28d-545939afea46 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:32.434048 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:32.457917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-41e7a50f-397b-4559-a28d-545939afea46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0fa16780b0b460f94a3ead14e186b01', 'request_id': 'req-41e7a50f-397b-4559-a28d-545939afea46'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:32.464045 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-41e7a50f-397b-4559-a28d-545939afea46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0fa16780b0b460f94a3ead14e186b01', 'request_id': 'req-41e7a50f-397b-4559-a28d-545939afea46'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:32.470564 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-41e7a50f-397b-4559-a28d-545939afea46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0fa16780b0b460f94a3ead14e186b01', 'request_id': 'req-41e7a50f-397b-4559-a28d-545939afea46'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:32.481098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-41e7a50f-397b-4559-a28d-545939afea46 admin admin] The token cache is being invalidate because project e0fa16780b0b460f94a3ead14e186b01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:42:32.481384 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-41e7a50f-397b-4559-a28d-545939afea46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:32.481997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-41e7a50f-397b-4559-a28d-545939afea46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:32.483456 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1143/2276] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:42:32 2022] DELETE /identity/v3/projects/e0fa16780b0b460f94a3ead14e186b01 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:32.488559 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8afd0-c36c-4c90-8528-0100470695ba service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:32.498337 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8afd0-c36c-4c90-8528-0100470695ba service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:32.557634 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168413a-0a4b-4b7e-890b-f6963f67a7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:32.558279 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168413a-0a4b-4b7e-890b-f6963f67a7a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:32.558562 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168413a-0a4b-4b7e-890b-f6963f67a7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:32.558840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168413a-0a4b-4b7e-890b-f6963f67a7a9 admin admin] PATH_INFO: `/v3/users/3650702d6adb4cf39b37cecd7de48dd5` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:32.559968 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5168413a-0a4b-4b7e-890b-f6963f67a7a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3650702d6adb4cf39b37cecd7de48dd5)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:32.561929 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5168413a-0a4b-4b7e-890b-f6963f67a7a9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:32.614816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5168413a-0a4b-4b7e-890b-f6963f67a7a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3650702d6adb4cf39b37cecd7de48dd5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:32.615478 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5168413a-0a4b-4b7e-890b-f6963f67a7a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3650702d6adb4cf39b37cecd7de48dd5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:32.623365 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5168413a-0a4b-4b7e-890b-f6963f67a7a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3650702d6adb4cf39b37cecd7de48dd5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:32.631921 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5168413a-0a4b-4b7e-890b-f6963f67a7a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3650702d6adb4cf39b37cecd7de48dd5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:32.632658 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5168413a-0a4b-4b7e-890b-f6963f67a7a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3650702d6adb4cf39b37cecd7de48dd5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:32.650207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1134/2277] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:42:32 2022] DELETE /identity/v3/users/3650702d6adb4cf39b37cecd7de48dd5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:32.945975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7a50f-397b-4559-a28d-545939afea46 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:32.947548 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-41e7a50f-397b-4559-a28d-545939afea46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:32.977440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7a50f-397b-4559-a28d-545939afea46 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:33.044708 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062968ec-8411-4261-9aa7-e4432e30dd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:33.045444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062968ec-8411-4261-9aa7-e4432e30dd1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:33.045846 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062968ec-8411-4261-9aa7-e4432e30dd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:33.046180 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062968ec-8411-4261-9aa7-e4432e30dd1a admin admin] PATH_INFO: `/v3/projects/8262b5cdea114008aea59b1f9a8f7e54` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:33.047361 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062968ec-8411-4261-9aa7-e4432e30dd1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8262b5cdea114008aea59b1f9a8f7e54)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:33.049842 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062968ec-8411-4261-9aa7-e4432e30dd1a admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:33.097849 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-062968ec-8411-4261-9aa7-e4432e30dd1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8262b5cdea114008aea59b1f9a8f7e54', 'request_id': 'req-062968ec-8411-4261-9aa7-e4432e30dd1a'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:33.104421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-062968ec-8411-4261-9aa7-e4432e30dd1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8262b5cdea114008aea59b1f9a8f7e54', 'request_id': 'req-062968ec-8411-4261-9aa7-e4432e30dd1a'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:33.114214 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-062968ec-8411-4261-9aa7-e4432e30dd1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8262b5cdea114008aea59b1f9a8f7e54', 'request_id': 'req-062968ec-8411-4261-9aa7-e4432e30dd1a'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:33.127068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-062968ec-8411-4261-9aa7-e4432e30dd1a admin admin] The token cache is being invalidate because project 8262b5cdea114008aea59b1f9a8f7e54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:42:33.127588 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-062968ec-8411-4261-9aa7-e4432e30dd1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:33.128304 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-062968ec-8411-4261-9aa7-e4432e30dd1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:33.133569 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1144/2278] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:42:32 2022] DELETE /identity/v3/projects/8262b5cdea114008aea59b1f9a8f7e54 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:33.136266 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26e8cb-dfa3-43fb-92f7-016c3fd0b642 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:33.136784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26e8cb-dfa3-43fb-92f7-016c3fd0b642 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:33.137300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26e8cb-dfa3-43fb-92f7-016c3fd0b642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:33.162195 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-cf26e8cb-dfa3-43fb-92f7-016c3fd0b642 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:33.163482 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cf26e8cb-dfa3-43fb-92f7-016c3fd0b642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:33.241332 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1135/2279] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:33.242981 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80326d7b-f9fc-4f36-b916-b62274489bf6 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:33.243771 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80326d7b-f9fc-4f36-b916-b62274489bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:33.244051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80326d7b-f9fc-4f36-b916-b62274489bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:33.266369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-80326d7b-f9fc-4f36-b916-b62274489bf6 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:33.268394 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-80326d7b-f9fc-4f36-b916-b62274489bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:33.281673 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1145/2280] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:42:33.287308 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26e8cb-dfa3-43fb-92f7-016c3fd0b642 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:33.298659 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26e8cb-dfa3-43fb-92f7-016c3fd0b642 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:33.319233 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4622e-a867-4602-a6ef-7d56db768477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:33.320205 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4622e-a867-4602-a6ef-7d56db768477 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:33.320778 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4622e-a867-4602-a6ef-7d56db768477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:33.321066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4622e-a867-4602-a6ef-7d56db768477 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:33.321625 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f4622e-a867-4602-a6ef-7d56db768477 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:42:33.321994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f4622e-a867-4602-a6ef-7d56db768477 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:33.323804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f4622e-a867-4602-a6ef-7d56db768477 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:33.338409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1136/2281] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:42:33.343631 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80326d7b-f9fc-4f36-b916-b62274489bf6 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:33.373137 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80326d7b-f9fc-4f36-b916-b62274489bf6 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:33.383787 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ec342-bbae-47f0-883a-2373a88071a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:33.385025 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ec342-bbae-47f0-883a-2373a88071a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:33.385400 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ec342-bbae-47f0-883a-2373a88071a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:33.385613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ec342-bbae-47f0-883a-2373a88071a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:33.389813 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ec342-bbae-47f0-883a-2373a88071a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:33.393228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ec342-bbae-47f0-883a-2373a88071a9 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:33.428342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1146/2282] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:42:33 2022] POST /identity/v3/projects => generated 371 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:42:33.429815 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4622e-a867-4602-a6ef-7d56db768477 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:33.434902 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:33.442547 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4622e-a867-4602-a6ef-7d56db768477 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:33.508763 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a096f-8be6-43dd-bcb2-64ed722357a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:33.509373 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a096f-8be6-43dd-bcb2-64ed722357a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:33.509654 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a096f-8be6-43dd-bcb2-64ed722357a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:33.510391 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a096f-8be6-43dd-bcb2-64ed722357a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:33.510569 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a096f-8be6-43dd-bcb2-64ed722357a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:33.512522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a096f-8be6-43dd-bcb2-64ed722357a6 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:33.538730 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1137/2283] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:42:33 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:42:33.539912 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ec342-bbae-47f0-883a-2373a88071a9 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:33.551404 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ec342-bbae-47f0-883a-2373a88071a9 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:33.558243 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091ee46-0415-4248-ac85-e6e34d5898d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:33.558857 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091ee46-0415-4248-ac85-e6e34d5898d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:33.559135 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091ee46-0415-4248-ac85-e6e34d5898d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:33.559407 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091ee46-0415-4248-ac85-e6e34d5898d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:33.560055 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4091ee46-0415-4248-ac85-e6e34d5898d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:33.561870 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4091ee46-0415-4248-ac85-e6e34d5898d2 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:33.572794 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1147/2284] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:33.576827 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a096f-8be6-43dd-bcb2-64ed722357a6 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:33.587673 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a096f-8be6-43dd-bcb2-64ed722357a6 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:33.592271 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ffc26-b87c-4c7d-ae13-a258fe94729c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:33.592871 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ffc26-b87c-4c7d-ae13-a258fe94729c admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:33.593166 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ffc26-b87c-4c7d-ae13-a258fe94729c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:33.593443 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ffc26-b87c-4c7d-ae13-a258fe94729c admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:33.594075 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ffc26-b87c-4c7d-ae13-a258fe94729c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:33.595725 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ffc26-b87c-4c7d-ae13-a258fe94729c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:33.606684 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1138/2285] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:33.610762 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091ee46-0415-4248-ac85-e6e34d5898d2 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:33.616296 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:33.622583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091ee46-0415-4248-ac85-e6e34d5898d2 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:33.629027 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad72d83-238f-49f6-9715-a50c000653ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:33.629711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad72d83-238f-49f6-9715-a50c000653ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:33.630018 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad72d83-238f-49f6-9715-a50c000653ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:33.630306 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad72d83-238f-49f6-9715-a50c000653ea admin admin] PATH_INFO: `/v3/projects/70556d4cdc8145fda2a9826761e9e20a/users/67d31e578dbc4b3abd842d3a755b5467/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:33.647197 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad72d83-238f-49f6-9715-a50c000653ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=70556d4cdc8145fda2a9826761e9e20a, user_id=67d31e578dbc4b3abd842d3a755b5467, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:33.650479 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad72d83-238f-49f6-9715-a50c000653ea admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:33.664160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1148/2286] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:33 2022] PUT /identity/v3/projects/70556d4cdc8145fda2a9826761e9e20a/users/67d31e578dbc4b3abd842d3a755b5467/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:33.675823 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903a902-fe01-4182-9e80-c1c1f3c83bfb None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:33.676109 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903a902-fe01-4182-9e80-c1c1f3c83bfb None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:33.676381 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a903a902-fe01-4182-9e80-c1c1f3c83bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:33.705376 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a903a902-fe01-4182-9e80-c1c1f3c83bfb None None] MFA Rules not processed for user `67d31e578dbc4b3abd842d3a755b5467`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:33.706662 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a903a902-fe01-4182-9e80-c1c1f3c83bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:33.769007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1139/2287] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:33.780519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad72d83-238f-49f6-9715-a50c000653ea admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:33.782223 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-0ad72d83-238f-49f6-9715-a50c000653ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:33.837960 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad72d83-238f-49f6-9715-a50c000653ea admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:33.841766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70048ffd-05ed-4037-aee5-c0e95eee515c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87b5093e963c4753928a8c6091afc52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:33.842397 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70048ffd-05ed-4037-aee5-c0e95eee515c service glance] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:33.842670 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70048ffd-05ed-4037-aee5-c0e95eee515c service glance] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:33.842937 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70048ffd-05ed-4037-aee5-c0e95eee515c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:33.851269 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70048ffd-05ed-4037-aee5-c0e95eee515c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:33.852693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70048ffd-05ed-4037-aee5-c0e95eee515c service glance] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:33.861936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1149/2288] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:42:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:42:41.913647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdad78-eb9c-4a0c-8345-c565697dc3c2 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:41.914213 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdad78-eb9c-4a0c-8345-c565697dc3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:41.914913 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdad78-eb9c-4a0c-8345-c565697dc3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:41.936366 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-4dfdad78-eb9c-4a0c-8345-c565697dc3c2 None None] MFA Rules not processed for user `85bb137d0fb74031a49a5c629b32c32b`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:41.937663 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4dfdad78-eb9c-4a0c-8345-c565697dc3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:41.993589 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1140/2289] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 20 13:42:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:42.016172 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70048ffd-05ed-4037-aee5-c0e95eee515c service glance] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:42.018372 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-70048ffd-05ed-4037-aee5-c0e95eee515c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:42.087650 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70048ffd-05ed-4037-aee5-c0e95eee515c service glance] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:42.092904 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccdf9e9-8fed-4b6f-8bea-a7c10a432e65 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9834211488354651803000c7e8fab80b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:42.093549 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdf9e9-8fed-4b6f-8bea-a7c10a432e65 service swift] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:42.093867 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdf9e9-8fed-4b6f-8bea-a7c10a432e65 service swift] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:42.094205 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdf9e9-8fed-4b6f-8bea-a7c10a432e65 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:42.103713 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccdf9e9-8fed-4b6f-8bea-a7c10a432e65 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:42.105377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccdf9e9-8fed-4b6f-8bea-a7c10a432e65 service swift] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:42.114737 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1150/2290] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 20 13:42:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 13:42:48.601767 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfdad78-eb9c-4a0c-8345-c565697dc3c2 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:48.603444 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4dfdad78-eb9c-4a0c-8345-c565697dc3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:48.624200 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:48.680510 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfdad78-eb9c-4a0c-8345-c565697dc3c2 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:48.685346 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a999c-42a5-415a-84c4-24a6a96d8533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:48.685975 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a999c-42a5-415a-84c4-24a6a96d8533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:48.686255 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a999c-42a5-415a-84c4-24a6a96d8533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:48.686539 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a999c-42a5-415a-84c4-24a6a96d8533 admin admin] PATH_INFO: `/v3/users/c3f016db213248a687b74bfa6af27e45` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:48.687681 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a999c-42a5-415a-84c4-24a6a96d8533 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3f016db213248a687b74bfa6af27e45)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:48.689950 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a999c-42a5-415a-84c4-24a6a96d8533 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:48.753506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-514a999c-42a5-415a-84c4-24a6a96d8533 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3f016db213248a687b74bfa6af27e45'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:48.754264 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-514a999c-42a5-415a-84c4-24a6a96d8533 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3f016db213248a687b74bfa6af27e45'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:48.761884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-514a999c-42a5-415a-84c4-24a6a96d8533 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3f016db213248a687b74bfa6af27e45'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:48.772045 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-514a999c-42a5-415a-84c4-24a6a96d8533 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3f016db213248a687b74bfa6af27e45'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:48.772843 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-514a999c-42a5-415a-84c4-24a6a96d8533 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3f016db213248a687b74bfa6af27e45'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:48.793389 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1141/2291] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:42:48 2022] DELETE /identity/v3/users/c3f016db213248a687b74bfa6af27e45 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:48.804078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2736895-3fea-489c-8e6a-063c0615d84a None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:48.805083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2736895-3fea-489c-8e6a-063c0615d84a None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:48.805592 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2736895-3fea-489c-8e6a-063c0615d84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:48.830546 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-d2736895-3fea-489c-8e6a-063c0615d84a None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:48.832152 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-d2736895-3fea-489c-8e6a-063c0615d84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:48.910423 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1151/2292] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:42:48.920057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a999c-42a5-415a-84c4-24a6a96d8533 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:48.922096 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-514a999c-42a5-415a-84c4-24a6a96d8533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:48.992226 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a999c-42a5-415a-84c4-24a6a96d8533 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:48.996251 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831fc715-116e-4879-b477-095bd38993d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:48.998072 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831fc715-116e-4879-b477-095bd38993d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:48.998382 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831fc715-116e-4879-b477-095bd38993d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:48.998955 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831fc715-116e-4879-b477-095bd38993d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:49.010918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831fc715-116e-4879-b477-095bd38993d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:49.013109 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831fc715-116e-4879-b477-095bd38993d4 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:49.027522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1142/2293] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:42:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:42:49.363673 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2736895-3fea-489c-8e6a-063c0615d84a None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:49.366324 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-d2736895-3fea-489c-8e6a-063c0615d84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:49.386926 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2736895-3fea-489c-8e6a-063c0615d84a None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:49.394000 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908838e-a915-4520-b020-4a70a7e6c885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:49.394788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908838e-a915-4520-b020-4a70a7e6c885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:49.395074 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908838e-a915-4520-b020-4a70a7e6c885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:49.395350 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908838e-a915-4520-b020-4a70a7e6c885 admin admin] PATH_INFO: `/v3/projects/e7895f3e04f344088f59ea5578bb27b0` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:49.396488 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e908838e-a915-4520-b020-4a70a7e6c885 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7895f3e04f344088f59ea5578bb27b0)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:49.398820 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e908838e-a915-4520-b020-4a70a7e6c885 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:49.407455 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:49.457902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-e908838e-a915-4520-b020-4a70a7e6c885 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7895f3e04f344088f59ea5578bb27b0', 'request_id': 'req-e908838e-a915-4520-b020-4a70a7e6c885'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:49.464808 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-e908838e-a915-4520-b020-4a70a7e6c885 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7895f3e04f344088f59ea5578bb27b0', 'request_id': 'req-e908838e-a915-4520-b020-4a70a7e6c885'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:49.474309 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-e908838e-a915-4520-b020-4a70a7e6c885 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7895f3e04f344088f59ea5578bb27b0', 'request_id': 'req-e908838e-a915-4520-b020-4a70a7e6c885'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:49.487211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-e908838e-a915-4520-b020-4a70a7e6c885 admin admin] The token cache is being invalidate because project e7895f3e04f344088f59ea5578bb27b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:42:49.487505 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-e908838e-a915-4520-b020-4a70a7e6c885 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:49.488169 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-e908838e-a915-4520-b020-4a70a7e6c885 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:49.489741 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1152/2294] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:42:49 2022] DELETE /identity/v3/projects/e7895f3e04f344088f59ea5578bb27b0 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:49.494576 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831fc715-116e-4879-b477-095bd38993d4 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:49.509697 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831fc715-116e-4879-b477-095bd38993d4 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:49.559073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:49.589609 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4299b8-18e5-46d8-9642-502823cb6476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:49.590171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4299b8-18e5-46d8-9642-502823cb6476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:49.590388 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4299b8-18e5-46d8-9642-502823cb6476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:49.590660 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4299b8-18e5-46d8-9642-502823cb6476 admin admin] PATH_INFO: `/v3/users/ef180bbab3884344ab205ab6a4056618` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:49.591975 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4299b8-18e5-46d8-9642-502823cb6476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef180bbab3884344ab205ab6a4056618)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:49.597723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4299b8-18e5-46d8-9642-502823cb6476 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:49.668534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ca4299b8-18e5-46d8-9642-502823cb6476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef180bbab3884344ab205ab6a4056618'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:49.669264 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ca4299b8-18e5-46d8-9642-502823cb6476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef180bbab3884344ab205ab6a4056618'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:49.680182 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ca4299b8-18e5-46d8-9642-502823cb6476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef180bbab3884344ab205ab6a4056618'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:49.691395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ca4299b8-18e5-46d8-9642-502823cb6476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef180bbab3884344ab205ab6a4056618'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:49.692394 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ca4299b8-18e5-46d8-9642-502823cb6476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef180bbab3884344ab205ab6a4056618'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:49.716376 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1143/2295] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:42:49 2022] DELETE /identity/v3/users/ef180bbab3884344ab205ab6a4056618 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:50.022352 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908838e-a915-4520-b020-4a70a7e6c885 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:50.025064 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-e908838e-a915-4520-b020-4a70a7e6c885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:50.049609 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908838e-a915-4520-b020-4a70a7e6c885 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:50.118854 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029aae2-01cd-46f0-adc1-51d6ff0fc099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:50.119997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029aae2-01cd-46f0-adc1-51d6ff0fc099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:50.120336 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029aae2-01cd-46f0-adc1-51d6ff0fc099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:50.120676 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029aae2-01cd-46f0-adc1-51d6ff0fc099 admin admin] PATH_INFO: `/v3/projects/ec9ab8a43bfd43f0979b785d84e4b6b5` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:50.121985 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2029aae2-01cd-46f0-adc1-51d6ff0fc099 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec9ab8a43bfd43f0979b785d84e4b6b5)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:50.124039 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2029aae2-01cd-46f0-adc1-51d6ff0fc099 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:50.186050 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-2029aae2-01cd-46f0-adc1-51d6ff0fc099 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec9ab8a43bfd43f0979b785d84e4b6b5', 'request_id': 'req-2029aae2-01cd-46f0-adc1-51d6ff0fc099'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:50.195839 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-2029aae2-01cd-46f0-adc1-51d6ff0fc099 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec9ab8a43bfd43f0979b785d84e4b6b5', 'request_id': 'req-2029aae2-01cd-46f0-adc1-51d6ff0fc099'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:50.203239 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-2029aae2-01cd-46f0-adc1-51d6ff0fc099 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec9ab8a43bfd43f0979b785d84e4b6b5', 'request_id': 'req-2029aae2-01cd-46f0-adc1-51d6ff0fc099'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:50.214200 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-2029aae2-01cd-46f0-adc1-51d6ff0fc099 admin admin] The token cache is being invalidate because project ec9ab8a43bfd43f0979b785d84e4b6b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:42:50.214518 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-2029aae2-01cd-46f0-adc1-51d6ff0fc099 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:50.215159 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-2029aae2-01cd-46f0-adc1-51d6ff0fc099 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:50.216642 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1153/2296] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:42:50 2022] DELETE /identity/v3/projects/ec9ab8a43bfd43f0979b785d84e4b6b5 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:50.221704 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7575d5-a83e-446a-a3df-c0089d55d681 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:50.222000 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7575d5-a83e-446a-a3df-c0089d55d681 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:50.222282 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7575d5-a83e-446a-a3df-c0089d55d681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:50.242025 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2c7575d5-a83e-446a-a3df-c0089d55d681 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:50.243507 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2c7575d5-a83e-446a-a3df-c0089d55d681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:50.312611 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1144/2297] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:50.321954 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6725b-eeba-4598-8137-fc7495050e1b None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:50.322260 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6725b-eeba-4598-8137-fc7495050e1b None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:50.322532 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6725b-eeba-4598-8137-fc7495050e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:50.341850 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-03a6725b-eeba-4598-8137-fc7495050e1b None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:50.343040 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-03a6725b-eeba-4598-8137-fc7495050e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:50.350929 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1154/2298] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:50.355182 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7575d5-a83e-446a-a3df-c0089d55d681 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:50.365533 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7575d5-a83e-446a-a3df-c0089d55d681 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:50.370397 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d9268-1480-4e68-a0e1-ba61d20e453f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:50.370994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d9268-1480-4e68-a0e1-ba61d20e453f admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:50.371331 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d9268-1480-4e68-a0e1-ba61d20e453f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:50.371615 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d9268-1480-4e68-a0e1-ba61d20e453f admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:50.372304 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d9268-1480-4e68-a0e1-ba61d20e453f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:42:50.372607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d9268-1480-4e68-a0e1-ba61d20e453f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:50.374278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d9268-1480-4e68-a0e1-ba61d20e453f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:50.387746 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1145/2299] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:42:50.397550 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6725b-eeba-4598-8137-fc7495050e1b None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:50.410197 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6725b-eeba-4598-8137-fc7495050e1b None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:50.415221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99bcd3-6934-498e-8fcb-a55cc2b5ee96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:50.415953 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99bcd3-6934-498e-8fcb-a55cc2b5ee96 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:50.416298 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99bcd3-6934-498e-8fcb-a55cc2b5ee96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:50.416632 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99bcd3-6934-498e-8fcb-a55cc2b5ee96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:50.417368 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99bcd3-6934-498e-8fcb-a55cc2b5ee96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:50.419472 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99bcd3-6934-498e-8fcb-a55cc2b5ee96 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:50.435518 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:50.450440 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1155/2300] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:42:50 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:42:50.454672 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d9268-1480-4e68-a0e1-ba61d20e453f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:50.464585 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d9268-1480-4e68-a0e1-ba61d20e453f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:50.512577 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:50.536550 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60012227-4159-41c7-8c04-a050140e22d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:50.537512 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60012227-4159-41c7-8c04-a050140e22d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:50.537853 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60012227-4159-41c7-8c04-a050140e22d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:50.538186 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60012227-4159-41c7-8c04-a050140e22d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:50.538733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60012227-4159-41c7-8c04-a050140e22d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:50.540717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60012227-4159-41c7-8c04-a050140e22d6 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:50.568299 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1146/2301] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:42:50 2022] POST /identity/v3/users => generated 285 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:42:50.572962 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99bcd3-6934-498e-8fcb-a55cc2b5ee96 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:50.595591 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99bcd3-6934-498e-8fcb-a55cc2b5ee96 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:50.604509 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d22cbf-46d2-4fc5-8abc-439b8c2319b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:50.605568 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d22cbf-46d2-4fc5-8abc-439b8c2319b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:50.605989 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d22cbf-46d2-4fc5-8abc-439b8c2319b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:50.606475 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d22cbf-46d2-4fc5-8abc-439b8c2319b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:50.607462 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d22cbf-46d2-4fc5-8abc-439b8c2319b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:50.609410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d22cbf-46d2-4fc5-8abc-439b8c2319b0 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:50.624290 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1156/2302] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:50.632972 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60012227-4159-41c7-8c04-a050140e22d6 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:50.659444 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60012227-4159-41c7-8c04-a050140e22d6 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:50.671363 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c1e54-00e8-48c4-9fc4-33d721743125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:50.672636 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c1e54-00e8-48c4-9fc4-33d721743125 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:50.673152 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c1e54-00e8-48c4-9fc4-33d721743125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:50.674962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c1e54-00e8-48c4-9fc4-33d721743125 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:50.676767 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c1e54-00e8-48c4-9fc4-33d721743125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:50.680297 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c1e54-00e8-48c4-9fc4-33d721743125 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:50.700941 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1147/2303] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:50.704369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d22cbf-46d2-4fc5-8abc-439b8c2319b0 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:50.718773 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d22cbf-46d2-4fc5-8abc-439b8c2319b0 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:50.729759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f8d34-ba3d-4beb-9c3a-0e975c381949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:50.729987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f8d34-ba3d-4beb-9c3a-0e975c381949 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:50.731945 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f8d34-ba3d-4beb-9c3a-0e975c381949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:50.732124 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f8d34-ba3d-4beb-9c3a-0e975c381949 admin admin] PATH_INFO: `/v3/projects/8c0b1ce7ba014c6a85d6036f354c5dff/users/ffc681a10964458c84bcd5fc84d84ec8/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:50.758217 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f8d34-ba3d-4beb-9c3a-0e975c381949 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c0b1ce7ba014c6a85d6036f354c5dff, user_id=ffc681a10964458c84bcd5fc84d84ec8, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:50.761201 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f8d34-ba3d-4beb-9c3a-0e975c381949 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:50.776991 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1157/2304] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:50 2022] PUT /identity/v3/projects/8c0b1ce7ba014c6a85d6036f354c5dff/users/ffc681a10964458c84bcd5fc84d84ec8/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:50.789473 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31db608-ec27-4d2c-a67b-6bc8963785ff None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:50.789687 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31db608-ec27-4d2c-a67b-6bc8963785ff None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:50.789957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31db608-ec27-4d2c-a67b-6bc8963785ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:50.823186 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c31db608-ec27-4d2c-a67b-6bc8963785ff None None] MFA Rules not processed for user `ffc681a10964458c84bcd5fc84d84ec8`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:50.824405 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c31db608-ec27-4d2c-a67b-6bc8963785ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:50.917327 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1148/2305] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:50.922414 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f8d34-ba3d-4beb-9c3a-0e975c381949 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:50.931271 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31db608-ec27-4d2c-a67b-6bc8963785ff None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:50.933177 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c31db608-ec27-4d2c-a67b-6bc8963785ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:50.999474 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f8d34-ba3d-4beb-9c3a-0e975c381949 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:51.000036 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31db608-ec27-4d2c-a67b-6bc8963785ff None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:51.004686 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317b8d3-007d-4470-9bbd-9551a7ca8ea0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:51.005583 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317b8d3-007d-4470-9bbd-9551a7ca8ea0 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:51.005759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bd4fc-7605-4d39-bd10-ce6fbc03dd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:51.006090 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317b8d3-007d-4470-9bbd-9551a7ca8ea0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:51.006372 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bd4fc-7605-4d39-bd10-ce6fbc03dd2a admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:51.006545 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317b8d3-007d-4470-9bbd-9551a7ca8ea0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:51.006666 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bd4fc-7605-4d39-bd10-ce6fbc03dd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:51.006912 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bd4fc-7605-4d39-bd10-ce6fbc03dd2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:51.007709 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0bd4fc-7605-4d39-bd10-ce6fbc03dd2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:51.008662 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-3317b8d3-007d-4470-9bbd-9551a7ca8ea0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:51.009865 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0bd4fc-7605-4d39-bd10-ce6fbc03dd2a admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:51.026514 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3317b8d3-007d-4470-9bbd-9551a7ca8ea0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:51.028086 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3317b8d3-007d-4470-9bbd-9551a7ca8ea0 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:51.043429 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1158/2306] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:42:50 2022] POST /identity/v3/projects => generated 367 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:42:51.047675 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1149/2307] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:42:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:42:51.049564 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bd4fc-7605-4d39-bd10-ce6fbc03dd2a admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:51.137876 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bd4fc-7605-4d39-bd10-ce6fbc03dd2a admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:51.143429 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c23584-e12c-44d3-a014-6fcf6d48b1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:51.144343 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c23584-e12c-44d3-a014-6fcf6d48b1e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:51.144772 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c23584-e12c-44d3-a014-6fcf6d48b1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:51.145293 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c23584-e12c-44d3-a014-6fcf6d48b1e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:51.146088 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c23584-e12c-44d3-a014-6fcf6d48b1e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:51.148433 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c23584-e12c-44d3-a014-6fcf6d48b1e7 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:51.184913 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1159/2308] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:42:51 2022] POST /identity/v3/users => generated 290 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:42:51.186251 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317b8d3-007d-4470-9bbd-9551a7ca8ea0 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:51.199366 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317b8d3-007d-4470-9bbd-9551a7ca8ea0 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:51.205492 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b1532-a88a-45a2-8ea8-f8c9614134a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:51.206332 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b1532-a88a-45a2-8ea8-f8c9614134a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:51.206612 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b1532-a88a-45a2-8ea8-f8c9614134a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:51.206886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b1532-a88a-45a2-8ea8-f8c9614134a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:51.207547 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b1532-a88a-45a2-8ea8-f8c9614134a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:51.209314 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b1532-a88a-45a2-8ea8-f8c9614134a3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:51.222580 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1150/2309] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:51.227123 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c23584-e12c-44d3-a014-6fcf6d48b1e7 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:51.247370 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c23584-e12c-44d3-a014-6fcf6d48b1e7 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:51.261639 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207324a-687b-4063-9bfa-6bd28e19a7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:51.262639 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207324a-687b-4063-9bfa-6bd28e19a7ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:51.263014 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207324a-687b-4063-9bfa-6bd28e19a7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:51.263386 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207324a-687b-4063-9bfa-6bd28e19a7ec admin admin] PATH_INFO: `/v3/projects/69e460249e664a9eaa14ddb1b812f9e7/users/867c62e65ee1497eb709cfd051b85db7/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:51.295937 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207324a-687b-4063-9bfa-6bd28e19a7ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=69e460249e664a9eaa14ddb1b812f9e7, user_id=867c62e65ee1497eb709cfd051b85db7, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:51.299554 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207324a-687b-4063-9bfa-6bd28e19a7ec admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:51.356425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b1532-a88a-45a2-8ea8-f8c9614134a3 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:51.357350 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1160/2310] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:51 2022] PUT /identity/v3/projects/69e460249e664a9eaa14ddb1b812f9e7/users/867c62e65ee1497eb709cfd051b85db7/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:51.374906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b1532-a88a-45a2-8ea8-f8c9614134a3 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:51.436691 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:51.457248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7733981c-c7f9-4da9-953b-7d6d41d831fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:51.458202 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733981c-c7f9-4da9-953b-7d6d41d831fe admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:51.458619 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733981c-c7f9-4da9-953b-7d6d41d831fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:51.458957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7733981c-c7f9-4da9-953b-7d6d41d831fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:51.459806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7733981c-c7f9-4da9-953b-7d6d41d831fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:51.461566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7733981c-c7f9-4da9-953b-7d6d41d831fe admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:51.479398 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1151/2311] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:51.481752 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207324a-687b-4063-9bfa-6bd28e19a7ec admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:51.499051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207324a-687b-4063-9bfa-6bd28e19a7ec admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:51.508510 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306809f-9682-4f4d-9074-a25c2fef6049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:51.509621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306809f-9682-4f4d-9074-a25c2fef6049 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:51.509621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306809f-9682-4f4d-9074-a25c2fef6049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:51.510059 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306809f-9682-4f4d-9074-a25c2fef6049 admin admin] PATH_INFO: `/v3/projects/69e460249e664a9eaa14ddb1b812f9e7/users/867c62e65ee1497eb709cfd051b85db7/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:51.512710 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306809f-9682-4f4d-9074-a25c2fef6049 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69e460249e664a9eaa14ddb1b812f9e7, user_id=867c62e65ee1497eb709cfd051b85db7, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:51.515660 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306809f-9682-4f4d-9074-a25c2fef6049 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:51.523643 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:51.541336 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1161/2312] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:51 2022] PUT /identity/v3/projects/69e460249e664a9eaa14ddb1b812f9e7/users/867c62e65ee1497eb709cfd051b85db7/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:51.543756 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b9a88-d44b-4767-9c6a-b72efd6f2915 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:51.544149 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b9a88-d44b-4767-9c6a-b72efd6f2915 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:51.545183 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b9a88-d44b-4767-9c6a-b72efd6f2915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:51.578888 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-987b9a88-d44b-4767-9c6a-b72efd6f2915 None None] MFA Rules not processed for user `867c62e65ee1497eb709cfd051b85db7`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:51.579184 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-987b9a88-d44b-4767-9c6a-b72efd6f2915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:51.685227 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306809f-9682-4f4d-9074-a25c2fef6049 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:51.685859 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1152/2313] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:51 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:51.687782 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b306809f-9682-4f4d-9074-a25c2fef6049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:51.761430 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306809f-9682-4f4d-9074-a25c2fef6049 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:51.772643 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c13c1-1364-428c-8e29-fe58bdb8ab4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:51.773503 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c13c1-1364-428c-8e29-fe58bdb8ab4d service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:51.773969 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c13c1-1364-428c-8e29-fe58bdb8ab4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:51.774402 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c13c1-1364-428c-8e29-fe58bdb8ab4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:51.786802 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c13c1-1364-428c-8e29-fe58bdb8ab4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:51.788725 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c13c1-1364-428c-8e29-fe58bdb8ab4d service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:51.800677 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1162/2314] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:42:51 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:42:53.969002 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b9a88-d44b-4767-9c6a-b72efd6f2915 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:54.086851 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b9a88-d44b-4767-9c6a-b72efd6f2915 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:54.093882 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24deb565-a517-4d59-8664-c8deb0d441fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:54.094758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24deb565-a517-4d59-8664-c8deb0d441fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:54.096456 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24deb565-a517-4d59-8664-c8deb0d441fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:54.096930 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24deb565-a517-4d59-8664-c8deb0d441fc admin admin] PATH_INFO: `/v3/users/ffc681a10964458c84bcd5fc84d84ec8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:54.098328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24deb565-a517-4d59-8664-c8deb0d441fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffc681a10964458c84bcd5fc84d84ec8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:54.101703 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24deb565-a517-4d59-8664-c8deb0d441fc admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:54.172507 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-24deb565-a517-4d59-8664-c8deb0d441fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffc681a10964458c84bcd5fc84d84ec8'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:54.173978 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-24deb565-a517-4d59-8664-c8deb0d441fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffc681a10964458c84bcd5fc84d84ec8'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:54.183670 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-24deb565-a517-4d59-8664-c8deb0d441fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffc681a10964458c84bcd5fc84d84ec8'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:54.201227 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-24deb565-a517-4d59-8664-c8deb0d441fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffc681a10964458c84bcd5fc84d84ec8'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:54.203538 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-24deb565-a517-4d59-8664-c8deb0d441fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffc681a10964458c84bcd5fc84d84ec8'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:54.213631 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:54.239183 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1153/2315] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:42:53 2022] DELETE /identity/v3/users/ffc681a10964458c84bcd5fc84d84ec8 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:54.246955 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db4c34-2f23-40e9-9129-10de1c5576e0 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:54.247475 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db4c34-2f23-40e9-9129-10de1c5576e0 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:54.247687 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db4c34-2f23-40e9-9129-10de1c5576e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:54.281719 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-00db4c34-2f23-40e9-9129-10de1c5576e0 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:54.283341 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-00db4c34-2f23-40e9-9129-10de1c5576e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:54.362393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1163/2316] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:54.372044 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24deb565-a517-4d59-8664-c8deb0d441fc admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:54.373799 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-24deb565-a517-4d59-8664-c8deb0d441fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:54.462181 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24deb565-a517-4d59-8664-c8deb0d441fc admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:54.468120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324ca2a-5a87-4fb7-968a-2822ac3df149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:54.470450 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324ca2a-5a87-4fb7-968a-2822ac3df149 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:54.471624 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324ca2a-5a87-4fb7-968a-2822ac3df149 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:54.472206 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324ca2a-5a87-4fb7-968a-2822ac3df149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:54.491853 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324ca2a-5a87-4fb7-968a-2822ac3df149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:54.500629 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324ca2a-5a87-4fb7-968a-2822ac3df149 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:54.512785 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1154/2317] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:42:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:42:55.391644 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db4c34-2f23-40e9-9129-10de1c5576e0 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:55.393717 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-00db4c34-2f23-40e9-9129-10de1c5576e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:55.415089 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db4c34-2f23-40e9-9129-10de1c5576e0 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:55.439638 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e1ade-3676-46ac-a6ee-574528844b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:55.441048 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e1ade-3676-46ac-a6ee-574528844b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:55.441834 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e1ade-3676-46ac-a6ee-574528844b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:55.442458 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e1ade-3676-46ac-a6ee-574528844b41 admin admin] PATH_INFO: `/v3/projects/8c0b1ce7ba014c6a85d6036f354c5dff` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:55.445809 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e1ade-3676-46ac-a6ee-574528844b41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c0b1ce7ba014c6a85d6036f354c5dff)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:55.448104 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e1ade-3676-46ac-a6ee-574528844b41 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:55.581111 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324ca2a-5a87-4fb7-968a-2822ac3df149 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:55.583599 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4324ca2a-5a87-4fb7-968a-2822ac3df149 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:55.585939 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-532e1ade-3676-46ac-a6ee-574528844b41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c0b1ce7ba014c6a85d6036f354c5dff', 'request_id': 'req-532e1ade-3676-46ac-a6ee-574528844b41'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:55.596177 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-532e1ade-3676-46ac-a6ee-574528844b41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c0b1ce7ba014c6a85d6036f354c5dff', 'request_id': 'req-532e1ade-3676-46ac-a6ee-574528844b41'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:55.607852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-532e1ade-3676-46ac-a6ee-574528844b41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c0b1ce7ba014c6a85d6036f354c5dff', 'request_id': 'req-532e1ade-3676-46ac-a6ee-574528844b41'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:55.614560 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324ca2a-5a87-4fb7-968a-2822ac3df149 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:55.624001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-532e1ade-3676-46ac-a6ee-574528844b41 admin admin] The token cache is being invalidate because project 8c0b1ce7ba014c6a85d6036f354c5dff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:42:55.624942 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-532e1ade-3676-46ac-a6ee-574528844b41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:55.625826 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-532e1ade-3676-46ac-a6ee-574528844b41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:55.631907 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1164/2318] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:42:55 2022] DELETE /identity/v3/projects/8c0b1ce7ba014c6a85d6036f354c5dff => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:55.632930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e1ade-3676-46ac-a6ee-574528844b41 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:55.638259 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-532e1ade-3676-46ac-a6ee-574528844b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:55.696619 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:55.714795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695a951-8bbc-491b-ac30-26facb5d7ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:55.715510 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695a951-8bbc-491b-ac30-26facb5d7ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:55.715935 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695a951-8bbc-491b-ac30-26facb5d7ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:55.716326 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695a951-8bbc-491b-ac30-26facb5d7ce6 admin admin] PATH_INFO: `/v3/users/55526ddbea014e6cbf7c079cb204059e` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:55.721152 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695a951-8bbc-491b-ac30-26facb5d7ce6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55526ddbea014e6cbf7c079cb204059e)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:55.724536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695a951-8bbc-491b-ac30-26facb5d7ce6 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:55.760275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e1ade-3676-46ac-a6ee-574528844b41 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:55.773523 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6286a-0ef4-4c98-b4a7-927a5594c59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:55.774299 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6286a-0ef4-4c98-b4a7-927a5594c59b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:55.774577 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6286a-0ef4-4c98-b4a7-927a5594c59b admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:55.775104 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6286a-0ef4-4c98-b4a7-927a5594c59b admin admin] PATH_INFO: `/v3/users/867c62e65ee1497eb709cfd051b85db7` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:55.777044 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6286a-0ef4-4c98-b4a7-927a5594c59b admin admin] RBAC: Authorizing `identity:delete_user(user_id=867c62e65ee1497eb709cfd051b85db7)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:55.779284 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6286a-0ef4-4c98-b4a7-927a5594c59b admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:55.791452 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5695a951-8bbc-491b-ac30-26facb5d7ce6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55526ddbea014e6cbf7c079cb204059e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:55.792317 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5695a951-8bbc-491b-ac30-26facb5d7ce6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55526ddbea014e6cbf7c079cb204059e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:55.801248 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5695a951-8bbc-491b-ac30-26facb5d7ce6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55526ddbea014e6cbf7c079cb204059e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:55.810826 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5695a951-8bbc-491b-ac30-26facb5d7ce6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55526ddbea014e6cbf7c079cb204059e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:55.811504 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5695a951-8bbc-491b-ac30-26facb5d7ce6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55526ddbea014e6cbf7c079cb204059e'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:55.840295 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1155/2319] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:42:55 2022] DELETE /identity/v3/users/55526ddbea014e6cbf7c079cb204059e => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:55.857828 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6cf6286a-0ef4-4c98-b4a7-927a5594c59b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '867c62e65ee1497eb709cfd051b85db7'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:55.858533 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6cf6286a-0ef4-4c98-b4a7-927a5594c59b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '867c62e65ee1497eb709cfd051b85db7'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:55.870283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6cf6286a-0ef4-4c98-b4a7-927a5594c59b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '867c62e65ee1497eb709cfd051b85db7'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:55.871631 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6cf6286a-0ef4-4c98-b4a7-927a5594c59b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '867c62e65ee1497eb709cfd051b85db7'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:55.882881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6cf6286a-0ef4-4c98-b4a7-927a5594c59b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '867c62e65ee1497eb709cfd051b85db7'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:55.915568 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1165/2320] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:42:55 2022] DELETE /identity/v3/users/867c62e65ee1497eb709cfd051b85db7 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:56.118373 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695a951-8bbc-491b-ac30-26facb5d7ce6 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:56.123672 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5695a951-8bbc-491b-ac30-26facb5d7ce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:56.139313 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:56.256152 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695a951-8bbc-491b-ac30-26facb5d7ce6 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:56.341818 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b312b-300b-484a-8eb7-e195910f84e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:56.342555 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b312b-300b-484a-8eb7-e195910f84e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:56.342837 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b312b-300b-484a-8eb7-e195910f84e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:56.343108 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b312b-300b-484a-8eb7-e195910f84e4 admin admin] PATH_INFO: `/v3/projects/f9addc326c51410f809805d8b44c5c17` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:56.344377 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b312b-300b-484a-8eb7-e195910f84e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9addc326c51410f809805d8b44c5c17)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:56.347458 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b312b-300b-484a-8eb7-e195910f84e4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:56.389946 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6286a-0ef4-4c98-b4a7-927a5594c59b admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:56.392329 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-6cf6286a-0ef4-4c98-b4a7-927a5594c59b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:56.436972 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6286a-0ef4-4c98-b4a7-927a5594c59b admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:56.442684 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-074b312b-300b-484a-8eb7-e195910f84e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9addc326c51410f809805d8b44c5c17', 'request_id': 'req-074b312b-300b-484a-8eb7-e195910f84e4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:56.462618 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-074b312b-300b-484a-8eb7-e195910f84e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9addc326c51410f809805d8b44c5c17', 'request_id': 'req-074b312b-300b-484a-8eb7-e195910f84e4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:56.481093 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-074b312b-300b-484a-8eb7-e195910f84e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9addc326c51410f809805d8b44c5c17', 'request_id': 'req-074b312b-300b-484a-8eb7-e195910f84e4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:56.519703 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-074b312b-300b-484a-8eb7-e195910f84e4 admin admin] The token cache is being invalidate because project f9addc326c51410f809805d8b44c5c17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:42:56.521228 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-074b312b-300b-484a-8eb7-e195910f84e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:56.522485 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-074b312b-300b-484a-8eb7-e195910f84e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:56.526648 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1156/2321] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:42:56 2022] DELETE /identity/v3/projects/f9addc326c51410f809805d8b44c5c17 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:56.531496 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d65916-8e4b-41af-ab2c-74b6e7edceb6 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:56.539556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d65916-8e4b-41af-ab2c-74b6e7edceb6 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:56.540092 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d65916-8e4b-41af-ab2c-74b6e7edceb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:56.567241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-41d65916-8e4b-41af-ab2c-74b6e7edceb6 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:56.568449 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-41d65916-8e4b-41af-ab2c-74b6e7edceb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:56.599690 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11b14b-4640-4e93-b2bb-d9d83481703c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:56.601892 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11b14b-4640-4e93-b2bb-d9d83481703c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:56.602170 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11b14b-4640-4e93-b2bb-d9d83481703c admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:56.602443 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11b14b-4640-4e93-b2bb-d9d83481703c admin admin] PATH_INFO: `/v3/projects/69e460249e664a9eaa14ddb1b812f9e7` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:56.605141 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11b14b-4640-4e93-b2bb-d9d83481703c admin admin] RBAC: Authorizing `identity:delete_project(project_id=69e460249e664a9eaa14ddb1b812f9e7)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:56.609327 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11b14b-4640-4e93-b2bb-d9d83481703c admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:56.640467 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:56.684789 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:56.691121 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-7d11b14b-4640-4e93-b2bb-d9d83481703c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69e460249e664a9eaa14ddb1b812f9e7', 'request_id': 'req-7d11b14b-4640-4e93-b2bb-d9d83481703c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:56.698432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1157/2322] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:42:56.706241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-7d11b14b-4640-4e93-b2bb-d9d83481703c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69e460249e664a9eaa14ddb1b812f9e7', 'request_id': 'req-7d11b14b-4640-4e93-b2bb-d9d83481703c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:56.708850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd3775-c7ba-4020-9e14-e299e17f1d9e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:56.709160 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd3775-c7ba-4020-9e14-e299e17f1d9e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:56.709573 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd3775-c7ba-4020-9e14-e299e17f1d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:56.723676 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-7d11b14b-4640-4e93-b2bb-d9d83481703c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69e460249e664a9eaa14ddb1b812f9e7', 'request_id': 'req-7d11b14b-4640-4e93-b2bb-d9d83481703c'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:56.744496 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-7d11b14b-4640-4e93-b2bb-d9d83481703c admin admin] The token cache is being invalidate because project 69e460249e664a9eaa14ddb1b812f9e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:42:56.744784 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-7d11b14b-4640-4e93-b2bb-d9d83481703c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:56.745692 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-7d11b14b-4640-4e93-b2bb-d9d83481703c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:42:56.747258 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1166/2323] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:42:56 2022] DELETE /identity/v3/projects/69e460249e664a9eaa14ddb1b812f9e7 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:42:56.750487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-82cd3775-c7ba-4020-9e14-e299e17f1d9e None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:56.752618 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742340bd-c580-489b-92a6-ed235241fdd4 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:56.752764 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-82cd3775-c7ba-4020-9e14-e299e17f1d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:56.753104 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742340bd-c580-489b-92a6-ed235241fdd4 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:56.753481 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742340bd-c580-489b-92a6-ed235241fdd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:56.786507 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-742340bd-c580-489b-92a6-ed235241fdd4 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:56.789072 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-742340bd-c580-489b-92a6-ed235241fdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:56.840544 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1158/2324] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:56.846536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd3775-c7ba-4020-9e14-e299e17f1d9e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:56.857222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd3775-c7ba-4020-9e14-e299e17f1d9e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:56.863575 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053e644-85fa-4663-b0e9-e7b167973bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:56.864348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053e644-85fa-4663-b0e9-e7b167973bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:56.864628 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053e644-85fa-4663-b0e9-e7b167973bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:56.864920 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053e644-85fa-4663-b0e9-e7b167973bc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:56.865577 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053e644-85fa-4663-b0e9-e7b167973bc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:42:56.865880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053e644-85fa-4663-b0e9-e7b167973bc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:56.867878 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053e644-85fa-4663-b0e9-e7b167973bc7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:56.871317 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1167/2325] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:56.878418 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370653f0-dc38-415c-9f21-b7bae9c069b6 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:56.878980 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370653f0-dc38-415c-9f21-b7bae9c069b6 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:56.879416 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370653f0-dc38-415c-9f21-b7bae9c069b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:56.882343 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1159/2326] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:42:56.886813 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053e644-85fa-4663-b0e9-e7b167973bc7 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:56.906317 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-370653f0-dc38-415c-9f21-b7bae9c069b6 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:56.909408 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053e644-85fa-4663-b0e9-e7b167973bc7 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:56.910086 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-370653f0-dc38-415c-9f21-b7bae9c069b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:56.918796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1168/2327] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:42:56.919590 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3c24f-4204-42bc-a796-899b4978f070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:56.920217 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3c24f-4204-42bc-a796-899b4978f070 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:56.920765 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3c24f-4204-42bc-a796-899b4978f070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:56.921082 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3c24f-4204-42bc-a796-899b4978f070 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:56.922112 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3c24f-4204-42bc-a796-899b4978f070 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:56.923755 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370653f0-dc38-415c-9f21-b7bae9c069b6 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:56.926115 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3c24f-4204-42bc-a796-899b4978f070 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:56.934936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370653f0-dc38-415c-9f21-b7bae9c069b6 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:56.942170 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b6cc5-6ac4-413a-b9fc-84b8bd61ef40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:56.943223 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b6cc5-6ac4-413a-b9fc-84b8bd61ef40 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:56.943739 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b6cc5-6ac4-413a-b9fc-84b8bd61ef40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:56.944143 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b6cc5-6ac4-413a-b9fc-84b8bd61ef40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:56.945441 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b6cc5-6ac4-413a-b9fc-84b8bd61ef40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:42:56.946294 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b6cc5-6ac4-413a-b9fc-84b8bd61ef40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:56.948976 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b6cc5-6ac4-413a-b9fc-84b8bd61ef40 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:56.964972 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1169/2328] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:42:56.969393 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b6cc5-6ac4-413a-b9fc-84b8bd61ef40 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:56.982039 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b6cc5-6ac4-413a-b9fc-84b8bd61ef40 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:56.983011 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1160/2329] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:42:56 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:42:56.987596 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3c24f-4204-42bc-a796-899b4978f070 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:57.059870 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c1b1c-01e9-4106-ac77-f11a1014c728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:57.060786 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c1b1c-01e9-4106-ac77-f11a1014c728 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:57.061286 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c1b1c-01e9-4106-ac77-f11a1014c728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:57.061756 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c1b1c-01e9-4106-ac77-f11a1014c728 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:57.062576 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c1b1c-01e9-4106-ac77-f11a1014c728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:57.065818 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c1b1c-01e9-4106-ac77-f11a1014c728 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:57.073827 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3c24f-4204-42bc-a796-899b4978f070 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:57.080429 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3326f666-fad5-4d82-b6fa-1fd1779d41f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:57.081266 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326f666-fad5-4d82-b6fa-1fd1779d41f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:57.081672 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326f666-fad5-4d82-b6fa-1fd1779d41f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:57.082179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326f666-fad5-4d82-b6fa-1fd1779d41f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:57.083053 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3326f666-fad5-4d82-b6fa-1fd1779d41f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:57.085181 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3326f666-fad5-4d82-b6fa-1fd1779d41f4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:57.109595 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1170/2330] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:42:56 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:42:57.113819 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c1b1c-01e9-4106-ac77-f11a1014c728 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:57.117080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1161/2331] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:42:56 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:42:57.118309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3326f666-fad5-4d82-b6fa-1fd1779d41f4 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:57.132100 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3326f666-fad5-4d82-b6fa-1fd1779d41f4 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:57.181194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:57.192465 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:57.210926 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c1b1c-01e9-4106-ac77-f11a1014c728 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:57.216200 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39390d4c-34bd-4254-ae0c-47e20f528f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:57.216835 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39390d4c-34bd-4254-ae0c-47e20f528f56 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:57.217400 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39390d4c-34bd-4254-ae0c-47e20f528f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:57.217616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39390d4c-34bd-4254-ae0c-47e20f528f56 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:57.218468 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39390d4c-34bd-4254-ae0c-47e20f528f56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:57.220910 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39390d4c-34bd-4254-ae0c-47e20f528f56 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:57.226190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991310c-3444-4b62-8d02-62b825a62091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:57.226882 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991310c-3444-4b62-8d02-62b825a62091 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:57.227269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991310c-3444-4b62-8d02-62b825a62091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:57.227607 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991310c-3444-4b62-8d02-62b825a62091 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:57.228325 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991310c-3444-4b62-8d02-62b825a62091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:57.230345 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991310c-3444-4b62-8d02-62b825a62091 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:57.245049 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1162/2332] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:57.248082 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1171/2333] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:42:57 2022] POST /identity/v3/users => generated 287 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:42:57.249327 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991310c-3444-4b62-8d02-62b825a62091 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:57.252768 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39390d4c-34bd-4254-ae0c-47e20f528f56 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:57.263093 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991310c-3444-4b62-8d02-62b825a62091 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:57.263297 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39390d4c-34bd-4254-ae0c-47e20f528f56 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:57.268302 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5605bb-b005-44ff-b03d-cf61c3d8f712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:57.268935 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5605bb-b005-44ff-b03d-cf61c3d8f712 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:57.269084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57663346-7068-4747-9b32-7a270d832184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:57.269227 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5605bb-b005-44ff-b03d-cf61c3d8f712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:57.269473 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5605bb-b005-44ff-b03d-cf61c3d8f712 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:57.270025 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57663346-7068-4747-9b32-7a270d832184 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:57.270146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5605bb-b005-44ff-b03d-cf61c3d8f712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:57.270477 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57663346-7068-4747-9b32-7a270d832184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:57.270873 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57663346-7068-4747-9b32-7a270d832184 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:57.271674 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5605bb-b005-44ff-b03d-cf61c3d8f712 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:57.271794 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57663346-7068-4747-9b32-7a270d832184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:57.273567 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57663346-7068-4747-9b32-7a270d832184 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:57.287883 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1163/2334] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:57.288107 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1172/2335] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:57.293210 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5605bb-b005-44ff-b03d-cf61c3d8f712 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:57.302331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57663346-7068-4747-9b32-7a270d832184 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:57.307253 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5605bb-b005-44ff-b03d-cf61c3d8f712 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:57.315381 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae98efa-fddb-4999-a7bd-eb896d760ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:57.315988 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae98efa-fddb-4999-a7bd-eb896d760ec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:57.316232 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57663346-7068-4747-9b32-7a270d832184 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:57.316336 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae98efa-fddb-4999-a7bd-eb896d760ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:57.316542 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae98efa-fddb-4999-a7bd-eb896d760ec4 admin admin] PATH_INFO: `/v3/projects/84e31f403c2e461ba6cb8f60d0216201/users/6804a0ef8eaf49e2b8c02e9d1795d708/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:57.321370 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9d567-c602-467b-b2cc-2f163a362145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:57.322114 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9d567-c602-467b-b2cc-2f163a362145 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:57.322486 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9d567-c602-467b-b2cc-2f163a362145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:57.322860 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9d567-c602-467b-b2cc-2f163a362145 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:57.323643 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec9d567-c602-467b-b2cc-2f163a362145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:57.325454 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec9d567-c602-467b-b2cc-2f163a362145 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:57.337445 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae98efa-fddb-4999-a7bd-eb896d760ec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84e31f403c2e461ba6cb8f60d0216201, user_id=6804a0ef8eaf49e2b8c02e9d1795d708, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:57.340546 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae98efa-fddb-4999-a7bd-eb896d760ec4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:57.341909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1173/2336] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:42:57.342866 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9d567-c602-467b-b2cc-2f163a362145 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:57.354124 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9d567-c602-467b-b2cc-2f163a362145 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:57.356729 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1164/2337] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:57 2022] PUT /identity/v3/projects/84e31f403c2e461ba6cb8f60d0216201/users/6804a0ef8eaf49e2b8c02e9d1795d708/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:57.368026 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f9291-53f5-4e6e-8575-b4a6d496096b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:57.368307 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f9291-53f5-4e6e-8575-b4a6d496096b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:57.368578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f9291-53f5-4e6e-8575-b4a6d496096b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:57.411719 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-385f9291-53f5-4e6e-8575-b4a6d496096b None None] MFA Rules not processed for user `6804a0ef8eaf49e2b8c02e9d1795d708`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:57.413006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-385f9291-53f5-4e6e-8575-b4a6d496096b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:57.440941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c540da-acc7-49b1-8a41-22d5df71fa88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:57.442751 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c540da-acc7-49b1-8a41-22d5df71fa88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:57.443313 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c540da-acc7-49b1-8a41-22d5df71fa88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:57.443846 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c540da-acc7-49b1-8a41-22d5df71fa88 admin admin] PATH_INFO: `/v3/projects/d15d68a115e44592aa896a797c2d0130/users/9062e47c8a22450db428554687f1f707/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:57.466863 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c540da-acc7-49b1-8a41-22d5df71fa88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d15d68a115e44592aa896a797c2d0130, user_id=9062e47c8a22450db428554687f1f707, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:57.469663 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c540da-acc7-49b1-8a41-22d5df71fa88 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:57.489426 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1174/2338] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:57 2022] PUT /identity/v3/projects/d15d68a115e44592aa896a797c2d0130/users/9062e47c8a22450db428554687f1f707/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:57.500302 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53880925-6c5c-40df-afc4-8a03fdd07f84 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:57.500560 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53880925-6c5c-40df-afc4-8a03fdd07f84 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:57.501490 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53880925-6c5c-40df-afc4-8a03fdd07f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:57.530679 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1165/2339] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:57.542941 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f9291-53f5-4e6e-8575-b4a6d496096b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:57.545472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-385f9291-53f5-4e6e-8575-b4a6d496096b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:57.555765 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-53880925-6c5c-40df-afc4-8a03fdd07f84 None None] MFA Rules not processed for user `9062e47c8a22450db428554687f1f707`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:57.559843 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-53880925-6c5c-40df-afc4-8a03fdd07f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:57.647665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1175/2340] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:42:57.652235 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53880925-6c5c-40df-afc4-8a03fdd07f84 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:57.656156 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f9291-53f5-4e6e-8575-b4a6d496096b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:57.690142 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:57.703868 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:57.777073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b023de9-fc2f-403c-8f43-f1ef53b37ba8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:57.777704 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b023de9-fc2f-403c-8f43-f1ef53b37ba8 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:57.777991 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b023de9-fc2f-403c-8f43-f1ef53b37ba8 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:57.778623 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b023de9-fc2f-403c-8f43-f1ef53b37ba8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:57.800842 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53880925-6c5c-40df-afc4-8a03fdd07f84 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:57.813136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd896ac1-8c4e-4ae1-96bb-fd38eb695b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:57.814968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd896ac1-8c4e-4ae1-96bb-fd38eb695b41 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:57.815470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd896ac1-8c4e-4ae1-96bb-fd38eb695b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:57.815807 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd896ac1-8c4e-4ae1-96bb-fd38eb695b41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:57.816845 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd896ac1-8c4e-4ae1-96bb-fd38eb695b41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:57.820799 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd896ac1-8c4e-4ae1-96bb-fd38eb695b41 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:57.859051 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b023de9-fc2f-403c-8f43-f1ef53b37ba8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:57.861154 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b023de9-fc2f-403c-8f43-f1ef53b37ba8 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:57.877735 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1166/2341] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:42:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:42:57.895197 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1176/2342] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:42:57 2022] POST /identity/v3/projects => generated 377 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:42:57.901166 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b023de9-fc2f-403c-8f43-f1ef53b37ba8 service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:57.925256 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b023de9-fc2f-403c-8f43-f1ef53b37ba8 service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:58.074593 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725e11f-e4b3-455e-8545-57a7599f0f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:58.075488 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725e11f-e4b3-455e-8545-57a7599f0f03 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:58.075831 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725e11f-e4b3-455e-8545-57a7599f0f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:58.076172 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725e11f-e4b3-455e-8545-57a7599f0f03 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:58.076888 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725e11f-e4b3-455e-8545-57a7599f0f03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:58.081108 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b725e11f-e4b3-455e-8545-57a7599f0f03 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:58.114021 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1167/2343] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:42:57 2022] POST /identity/v3/users => generated 295 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:42:58.115052 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd896ac1-8c4e-4ae1-96bb-fd38eb695b41 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:58.129463 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd896ac1-8c4e-4ae1-96bb-fd38eb695b41 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:58.137195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0455cc9-a4fe-4045-81bd-e2021e51ec89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:58.138420 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0455cc9-a4fe-4045-81bd-e2021e51ec89 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:58.139106 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0455cc9-a4fe-4045-81bd-e2021e51ec89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:58.139613 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0455cc9-a4fe-4045-81bd-e2021e51ec89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:58.140591 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0455cc9-a4fe-4045-81bd-e2021e51ec89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:58.142731 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0455cc9-a4fe-4045-81bd-e2021e51ec89 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:58.155760 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1177/2344] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:42:58.166153 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725e11f-e4b3-455e-8545-57a7599f0f03 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:58.192104 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725e11f-e4b3-455e-8545-57a7599f0f03 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:58.200013 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea5b2a-3da4-4be3-bdc2-12b69670b697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:58.200791 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea5b2a-3da4-4be3-bdc2-12b69670b697 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:58.201217 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea5b2a-3da4-4be3-bdc2-12b69670b697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:58.201499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea5b2a-3da4-4be3-bdc2-12b69670b697 admin admin] PATH_INFO: `/v3/projects/d7e35c2ecf9244f39f909da7b43cc29d/users/c767f5f5732e4fafaf9a5576f3683d67/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:58.221922 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea5b2a-3da4-4be3-bdc2-12b69670b697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7e35c2ecf9244f39f909da7b43cc29d, user_id=c767f5f5732e4fafaf9a5576f3683d67, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:58.224992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea5b2a-3da4-4be3-bdc2-12b69670b697 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:58.241120 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1168/2345] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:58 2022] PUT /identity/v3/projects/d7e35c2ecf9244f39f909da7b43cc29d/users/c767f5f5732e4fafaf9a5576f3683d67/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:58.244428 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0455cc9-a4fe-4045-81bd-e2021e51ec89 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:58.257860 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0455cc9-a4fe-4045-81bd-e2021e51ec89 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:58.349077 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bbdf7-9d61-497e-a905-15273762a4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:58.349982 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bbdf7-9d61-497e-a905-15273762a4d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:58.350378 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bbdf7-9d61-497e-a905-15273762a4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:58.350957 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bbdf7-9d61-497e-a905-15273762a4d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:58.351764 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3bbdf7-9d61-497e-a905-15273762a4d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:58.353480 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3bbdf7-9d61-497e-a905-15273762a4d8 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:58.371071 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1178/2346] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:42:58.373155 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea5b2a-3da4-4be3-bdc2-12b69670b697 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:58.388643 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea5b2a-3da4-4be3-bdc2-12b69670b697 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:58.396067 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4dfa6e-f9e5-463f-ac6d-be62978ddd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:58.396728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4dfa6e-f9e5-463f-ac6d-be62978ddd19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:58.397041 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4dfa6e-f9e5-463f-ac6d-be62978ddd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:58.397309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4dfa6e-f9e5-463f-ac6d-be62978ddd19 admin admin] PATH_INFO: `/v3/projects/d7e35c2ecf9244f39f909da7b43cc29d/users/c767f5f5732e4fafaf9a5576f3683d67/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:58.402206 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4dfa6e-f9e5-463f-ac6d-be62978ddd19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7e35c2ecf9244f39f909da7b43cc29d, user_id=c767f5f5732e4fafaf9a5576f3683d67, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:58.405022 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4dfa6e-f9e5-463f-ac6d-be62978ddd19 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:58.421026 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1169/2347] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:42:58 2022] PUT /identity/v3/projects/d7e35c2ecf9244f39f909da7b43cc29d/users/c767f5f5732e4fafaf9a5576f3683d67/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:42:58.430249 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bee37-ef74-4268-87ff-e315befb4a86 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:58.430770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bee37-ef74-4268-87ff-e315befb4a86 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:58.431232 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bee37-ef74-4268-87ff-e315befb4a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:58.474902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-267bee37-ef74-4268-87ff-e315befb4a86 None None] MFA Rules not processed for user `c767f5f5732e4fafaf9a5576f3683d67`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:42:58.476107 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-267bee37-ef74-4268-87ff-e315befb4a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:58.581894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1179/2348] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:42:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:42:58.593449 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4dfa6e-f9e5-463f-ac6d-be62978ddd19 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:42:58.595368 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8f4dfa6e-f9e5-463f-ac6d-be62978ddd19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:42:58.620051 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:42:58.661180 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4dfa6e-f9e5-463f-ac6d-be62978ddd19 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:42:58.665062 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada634d-2353-470e-9076-431cdcf5dc05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:42:58.665771 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada634d-2353-470e-9076-431cdcf5dc05 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:42:58.666124 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada634d-2353-470e-9076-431cdcf5dc05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:42:58.666533 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada634d-2353-470e-9076-431cdcf5dc05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:42:58.728047 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada634d-2353-470e-9076-431cdcf5dc05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:42:58.729769 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada634d-2353-470e-9076-431cdcf5dc05 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:42:58.739281 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1170/2349] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:42:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:43:08.578018 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=92,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:08.734650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=40 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:13.209418 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26063322-7ddd-4416-a3f3-7119c4feb20e None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:13.209975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26063322-7ddd-4416-a3f3-7119c4feb20e None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:13.209975 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26063322-7ddd-4416-a3f3-7119c4feb20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:13.230371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-26063322-7ddd-4416-a3f3-7119c4feb20e None None] MFA Rules not processed for user `85bb137d0fb74031a49a5c629b32c32b`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:13.231725 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-26063322-7ddd-4416-a3f3-7119c4feb20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:13.307682 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1180/2350] 10.222.0.23 () {58 vars in 967 bytes} [Thu Oct 20 13:43:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:13.318035 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada634d-2353-470e-9076-431cdcf5dc05 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:13.319518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0ada634d-2353-470e-9076-431cdcf5dc05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:13.400656 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada634d-2353-470e-9076-431cdcf5dc05 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:13.406030 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ec7e1-2f4d-48f7-8c8c-aecef988cb29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9834211488354651803000c7e8fab80b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:13.406646 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ec7e1-2f4d-48f7-8c8c-aecef988cb29 service swift] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:13.406925 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ec7e1-2f4d-48f7-8c8c-aecef988cb29 service swift] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:13.407199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ec7e1-2f4d-48f7-8c8c-aecef988cb29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:13.416924 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ec7e1-2f4d-48f7-8c8c-aecef988cb29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:13.418501 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ec7e1-2f4d-48f7-8c8c-aecef988cb29 service swift] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:13.427127 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1171/2351] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 20 13:43:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 13:43:13.955733 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26063322-7ddd-4416-a3f3-7119c4feb20e None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:13.965155 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26063322-7ddd-4416-a3f3-7119c4feb20e None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:13.969280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe24fb8d-31ee-4fad-9179-f6ab9cec1c3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:13.971443 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24fb8d-31ee-4fad-9179-f6ab9cec1c3b service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:13.971767 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24fb8d-31ee-4fad-9179-f6ab9cec1c3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:13.972045 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe24fb8d-31ee-4fad-9179-f6ab9cec1c3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:13.982371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24fb8d-31ee-4fad-9179-f6ab9cec1c3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:13.983922 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe24fb8d-31ee-4fad-9179-f6ab9cec1c3b service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:13.994698 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1181/2352] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:43:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:43:14.615404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ec7e1-2f4d-48f7-8c8c-aecef988cb29 service swift] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:14.683440 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ec7e1-2f4d-48f7-8c8c-aecef988cb29 service swift] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:14.687945 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176b4da-dd5f-4551-9ac8-0729947d9c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:14.688618 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176b4da-dd5f-4551-9ac8-0729947d9c3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:14.688995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176b4da-dd5f-4551-9ac8-0729947d9c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:14.689273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176b4da-dd5f-4551-9ac8-0729947d9c3f admin admin] PATH_INFO: `/v3/users/6804a0ef8eaf49e2b8c02e9d1795d708` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:14.690382 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176b4da-dd5f-4551-9ac8-0729947d9c3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6804a0ef8eaf49e2b8c02e9d1795d708)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:14.692278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176b4da-dd5f-4551-9ac8-0729947d9c3f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:14.738526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0176b4da-dd5f-4551-9ac8-0729947d9c3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6804a0ef8eaf49e2b8c02e9d1795d708'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:14.739189 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0176b4da-dd5f-4551-9ac8-0729947d9c3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6804a0ef8eaf49e2b8c02e9d1795d708'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:14.746351 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0176b4da-dd5f-4551-9ac8-0729947d9c3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6804a0ef8eaf49e2b8c02e9d1795d708'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:14.753538 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0176b4da-dd5f-4551-9ac8-0729947d9c3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6804a0ef8eaf49e2b8c02e9d1795d708'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:14.754159 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-0176b4da-dd5f-4551-9ac8-0729947d9c3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6804a0ef8eaf49e2b8c02e9d1795d708'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:14.770994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1172/2353] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:43:14 2022] DELETE /identity/v3/users/6804a0ef8eaf49e2b8c02e9d1795d708 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:14.775595 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47d5ed-85ab-466a-8025-8934713aa104 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:14.776051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47d5ed-85ab-466a-8025-8934713aa104 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:14.776162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47d5ed-85ab-466a-8025-8934713aa104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:14.795084 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-7c47d5ed-85ab-466a-8025-8934713aa104 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:14.796270 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-7c47d5ed-85ab-466a-8025-8934713aa104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:14.854834 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1182/2354] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:14.865394 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176b4da-dd5f-4551-9ac8-0729947d9c3f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:14.867058 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0176b4da-dd5f-4551-9ac8-0729947d9c3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:14.879240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:14.929446 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176b4da-dd5f-4551-9ac8-0729947d9c3f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:14.933146 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728edef-edbe-42a7-9205-8be21661be72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:14.933749 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728edef-edbe-42a7-9205-8be21661be72 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:14.934057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728edef-edbe-42a7-9205-8be21661be72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:14.934332 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728edef-edbe-42a7-9205-8be21661be72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:14.944311 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c728edef-edbe-42a7-9205-8be21661be72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:14.945780 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c728edef-edbe-42a7-9205-8be21661be72 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:14.955840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1173/2355] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:43:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:43:15.225899 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47d5ed-85ab-466a-8025-8934713aa104 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:15.227399 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-7c47d5ed-85ab-466a-8025-8934713aa104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:15.243266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47d5ed-85ab-466a-8025-8934713aa104 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:15.248432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755deaa-c8a9-48e7-a018-ce7437507302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:15.249187 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755deaa-c8a9-48e7-a018-ce7437507302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:15.249581 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755deaa-c8a9-48e7-a018-ce7437507302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:15.249877 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755deaa-c8a9-48e7-a018-ce7437507302 admin admin] PATH_INFO: `/v3/projects/84e31f403c2e461ba6cb8f60d0216201` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:15.251328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755deaa-c8a9-48e7-a018-ce7437507302 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84e31f403c2e461ba6cb8f60d0216201)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:15.253825 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755deaa-c8a9-48e7-a018-ce7437507302 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:15.304282 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5755deaa-c8a9-48e7-a018-ce7437507302 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84e31f403c2e461ba6cb8f60d0216201', 'request_id': 'req-5755deaa-c8a9-48e7-a018-ce7437507302'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:15.310898 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5755deaa-c8a9-48e7-a018-ce7437507302 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84e31f403c2e461ba6cb8f60d0216201', 'request_id': 'req-5755deaa-c8a9-48e7-a018-ce7437507302'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:15.319574 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5755deaa-c8a9-48e7-a018-ce7437507302 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84e31f403c2e461ba6cb8f60d0216201', 'request_id': 'req-5755deaa-c8a9-48e7-a018-ce7437507302'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:15.332136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5755deaa-c8a9-48e7-a018-ce7437507302 admin admin] The token cache is being invalidate because project 84e31f403c2e461ba6cb8f60d0216201 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:43:15.332424 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5755deaa-c8a9-48e7-a018-ce7437507302 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:15.333099 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5755deaa-c8a9-48e7-a018-ce7437507302 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:15.334595 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1183/2356] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:43:15 2022] DELETE /identity/v3/projects/84e31f403c2e461ba6cb8f60d0216201 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:15.339387 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34dc607-7eac-4beb-8d0f-e9e6fe663832 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:15.339670 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34dc607-7eac-4beb-8d0f-e9e6fe663832 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:15.339945 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34dc607-7eac-4beb-8d0f-e9e6fe663832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:15.364797 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a34dc607-7eac-4beb-8d0f-e9e6fe663832 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:15.367222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a34dc607-7eac-4beb-8d0f-e9e6fe663832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:15.390932 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755deaa-c8a9-48e7-a018-ce7437507302 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:15.393637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-5755deaa-c8a9-48e7-a018-ce7437507302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:15.421226 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:15.452668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1174/2357] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:15.462411 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188d6d5-4df4-4039-8e0d-3a2f257a3727 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:15.462710 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188d6d5-4df4-4039-8e0d-3a2f257a3727 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:15.462991 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188d6d5-4df4-4039-8e0d-3a2f257a3727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:15.478925 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755deaa-c8a9-48e7-a018-ce7437507302 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:15.483484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9188d6d5-4df4-4039-8e0d-3a2f257a3727 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:15.484852 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9188d6d5-4df4-4039-8e0d-3a2f257a3727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:15.485519 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf4811-330e-4c43-8388-f076fe9df85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:15.486498 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf4811-330e-4c43-8388-f076fe9df85c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:15.486875 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf4811-330e-4c43-8388-f076fe9df85c admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:15.487150 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf4811-330e-4c43-8388-f076fe9df85c admin admin] PATH_INFO: `/v3/users/69c04718cd82496abbc82e353873e604` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:15.488330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf4811-330e-4c43-8388-f076fe9df85c admin admin] RBAC: Authorizing `identity:delete_user(user_id=69c04718cd82496abbc82e353873e604)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:15.490283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf4811-330e-4c43-8388-f076fe9df85c admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:15.497126 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1175/2358] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:15.499699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188d6d5-4df4-4039-8e0d-3a2f257a3727 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:15.510517 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188d6d5-4df4-4039-8e0d-3a2f257a3727 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:15.516799 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d40cfb-df6c-4b5f-94f0-988761e62bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:15.517647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d40cfb-df6c-4b5f-94f0-988761e62bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:15.518216 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d40cfb-df6c-4b5f-94f0-988761e62bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:15.518431 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d40cfb-df6c-4b5f-94f0-988761e62bc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:15.519191 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d40cfb-df6c-4b5f-94f0-988761e62bc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:43:15.519591 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d40cfb-df6c-4b5f-94f0-988761e62bc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:15.521515 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d40cfb-df6c-4b5f-94f0-988761e62bc1 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:15.540204 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1176/2359] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:43:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:43:15.546605 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d40cfb-df6c-4b5f-94f0-988761e62bc1 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:15.558568 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d40cfb-df6c-4b5f-94f0-988761e62bc1 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:15.564131 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df418a3-4706-4530-9bb3-54b115cbd388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:15.565422 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df418a3-4706-4530-9bb3-54b115cbd388 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:15.565705 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df418a3-4706-4530-9bb3-54b115cbd388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:15.565981 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df418a3-4706-4530-9bb3-54b115cbd388 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:15.566623 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df418a3-4706-4530-9bb3-54b115cbd388 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:15.568522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df418a3-4706-4530-9bb3-54b115cbd388 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:15.573949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-32bf4811-330e-4c43-8388-f076fe9df85c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69c04718cd82496abbc82e353873e604'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:15.575356 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-32bf4811-330e-4c43-8388-f076fe9df85c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69c04718cd82496abbc82e353873e604'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:15.585793 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-32bf4811-330e-4c43-8388-f076fe9df85c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69c04718cd82496abbc82e353873e604'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:15.586722 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-32bf4811-330e-4c43-8388-f076fe9df85c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69c04718cd82496abbc82e353873e604'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:15.596453 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-32bf4811-330e-4c43-8388-f076fe9df85c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69c04718cd82496abbc82e353873e604'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:15.601398 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1177/2360] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:43:15 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:43:15.605211 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df418a3-4706-4530-9bb3-54b115cbd388 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:15.607439 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2df418a3-4706-4530-9bb3-54b115cbd388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:15.619009 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1184/2361] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:43:15 2022] DELETE /identity/v3/users/69c04718cd82496abbc82e353873e604 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:43:15.695165 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df418a3-4706-4530-9bb3-54b115cbd388 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:15.732506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:15.769298 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390e0d5-580c-4f48-8c95-b4bfc6467ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:15.769918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390e0d5-580c-4f48-8c95-b4bfc6467ef6 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:15.770203 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390e0d5-580c-4f48-8c95-b4bfc6467ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:15.770479 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390e0d5-580c-4f48-8c95-b4bfc6467ef6 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:15.771126 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390e0d5-580c-4f48-8c95-b4bfc6467ef6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:15.773154 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390e0d5-580c-4f48-8c95-b4bfc6467ef6 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:15.800509 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1178/2362] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:43:15 2022] POST /identity/v3/users => generated 287 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:43:15.801964 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf4811-330e-4c43-8388-f076fe9df85c admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:15.812323 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf4811-330e-4c43-8388-f076fe9df85c admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:15.821512 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7e4c5-3423-4692-b89f-9b6688885bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:15.822220 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7e4c5-3423-4692-b89f-9b6688885bab admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:15.822687 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7e4c5-3423-4692-b89f-9b6688885bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:15.823051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7e4c5-3423-4692-b89f-9b6688885bab admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:15.823802 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7e4c5-3423-4692-b89f-9b6688885bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:15.825849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390e0d5-580c-4f48-8c95-b4bfc6467ef6 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:15.825978 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7e4c5-3423-4692-b89f-9b6688885bab admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:15.827941 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2390e0d5-580c-4f48-8c95-b4bfc6467ef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:15.839040 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1185/2363] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:43:15.840622 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390e0d5-580c-4f48-8c95-b4bfc6467ef6 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:15.842785 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7e4c5-3423-4692-b89f-9b6688885bab admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:15.854329 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7e4c5-3423-4692-b89f-9b6688885bab admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:15.856595 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223dd353-cc84-451d-978b-f527fe737234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:15.857790 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223dd353-cc84-451d-978b-f527fe737234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:15.858577 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223dd353-cc84-451d-978b-f527fe737234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:15.858862 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223dd353-cc84-451d-978b-f527fe737234 admin admin] PATH_INFO: `/v3/projects/a444b7fa425b4f0ba7833b93607daa25` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:15.860265 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61ec9e-8ca0-47e7-96f0-ca5f988460b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:15.860839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223dd353-cc84-451d-978b-f527fe737234 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a444b7fa425b4f0ba7833b93607daa25)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:15.861234 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61ec9e-8ca0-47e7-96f0-ca5f988460b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:15.861506 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61ec9e-8ca0-47e7-96f0-ca5f988460b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:15.861773 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61ec9e-8ca0-47e7-96f0-ca5f988460b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:15.862439 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61ec9e-8ca0-47e7-96f0-ca5f988460b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:15.862858 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223dd353-cc84-451d-978b-f527fe737234 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:15.864014 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61ec9e-8ca0-47e7-96f0-ca5f988460b8 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:15.874721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1186/2364] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:43:15.878871 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61ec9e-8ca0-47e7-96f0-ca5f988460b8 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:15.888497 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61ec9e-8ca0-47e7-96f0-ca5f988460b8 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:15.894049 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ce87a-5b46-42e0-8394-0a8dc2206956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:15.894746 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ce87a-5b46-42e0-8394-0a8dc2206956 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:15.895108 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ce87a-5b46-42e0-8394-0a8dc2206956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:15.895384 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ce87a-5b46-42e0-8394-0a8dc2206956 admin admin] PATH_INFO: `/v3/projects/9b0a0dff64c64661a05db93ee8a8fd7d/users/704f8cb826214bc78dd49d0962f1417a/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:15.911881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ce87a-5b46-42e0-8394-0a8dc2206956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b0a0dff64c64661a05db93ee8a8fd7d, user_id=704f8cb826214bc78dd49d0962f1417a, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:15.914273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-223dd353-cc84-451d-978b-f527fe737234 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a444b7fa425b4f0ba7833b93607daa25', 'request_id': 'req-223dd353-cc84-451d-978b-f527fe737234'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:15.914621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ce87a-5b46-42e0-8394-0a8dc2206956 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:15.921997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-223dd353-cc84-451d-978b-f527fe737234 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a444b7fa425b4f0ba7833b93607daa25', 'request_id': 'req-223dd353-cc84-451d-978b-f527fe737234'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:15.928437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1187/2365] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:43:15 2022] PUT /identity/v3/projects/9b0a0dff64c64661a05db93ee8a8fd7d/users/704f8cb826214bc78dd49d0962f1417a/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:43:15.930553 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-223dd353-cc84-451d-978b-f527fe737234 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a444b7fa425b4f0ba7833b93607daa25', 'request_id': 'req-223dd353-cc84-451d-978b-f527fe737234'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:15.939693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac5f9f-89ec-4339-8462-913e5af2fba4 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:15.939984 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac5f9f-89ec-4339-8462-913e5af2fba4 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:15.940260 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac5f9f-89ec-4339-8462-913e5af2fba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:15.942667 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-223dd353-cc84-451d-978b-f527fe737234 admin admin] The token cache is being invalidate because project a444b7fa425b4f0ba7833b93607daa25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:43:15.942956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-223dd353-cc84-451d-978b-f527fe737234 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:15.943667 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-223dd353-cc84-451d-978b-f527fe737234 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:15.945179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1179/2366] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:43:15 2022] DELETE /identity/v3/projects/a444b7fa425b4f0ba7833b93607daa25 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:43:15.949648 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f237a1-0da8-46c0-bc4f-8d9b06afd839 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:15.949997 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f237a1-0da8-46c0-bc4f-8d9b06afd839 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:15.950334 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f237a1-0da8-46c0-bc4f-8d9b06afd839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:15.971183 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-cfac5f9f-89ec-4339-8462-913e5af2fba4 None None] MFA Rules not processed for user `704f8cb826214bc78dd49d0962f1417a`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:15.972409 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-cfac5f9f-89ec-4339-8462-913e5af2fba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:15.972591 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a0f237a1-0da8-46c0-bc4f-8d9b06afd839 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:15.973797 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a0f237a1-0da8-46c0-bc4f-8d9b06afd839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:16.010792 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:16.034397 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1180/2367] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:16.038692 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1188/2368] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:43:16.041494 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426b3e6-b063-4f5f-8d8a-85a01e000c36 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:16.041777 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426b3e6-b063-4f5f-8d8a-85a01e000c36 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:16.042049 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426b3e6-b063-4f5f-8d8a-85a01e000c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:16.051085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac5f9f-89ec-4339-8462-913e5af2fba4 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:16.052522 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-cfac5f9f-89ec-4339-8462-913e5af2fba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:16.061915 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-6426b3e6-b063-4f5f-8d8a-85a01e000c36 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:16.063071 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6426b3e6-b063-4f5f-8d8a-85a01e000c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:16.071450 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1181/2369] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:43:16.075469 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426b3e6-b063-4f5f-8d8a-85a01e000c36 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:16.084733 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426b3e6-b063-4f5f-8d8a-85a01e000c36 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:16.089339 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b7de1-2093-43e4-a1c8-8d92dadbed06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:16.089917 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b7de1-2093-43e4-a1c8-8d92dadbed06 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:16.090191 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b7de1-2093-43e4-a1c8-8d92dadbed06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:16.090459 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b7de1-2093-43e4-a1c8-8d92dadbed06 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:16.091094 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b7de1-2093-43e4-a1c8-8d92dadbed06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:43:16.091392 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b7de1-2093-43e4-a1c8-8d92dadbed06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:16.092980 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b7de1-2093-43e4-a1c8-8d92dadbed06 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:16.104454 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1182/2370] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:43:16.108431 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b7de1-2093-43e4-a1c8-8d92dadbed06 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:16.118179 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b7de1-2093-43e4-a1c8-8d92dadbed06 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:16.123457 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858b7e6-4a57-4064-85cf-aae3f9a6518d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:16.124033 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858b7e6-4a57-4064-85cf-aae3f9a6518d admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:16.124309 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858b7e6-4a57-4064-85cf-aae3f9a6518d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:16.124583 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858b7e6-4a57-4064-85cf-aae3f9a6518d admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:16.125237 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858b7e6-4a57-4064-85cf-aae3f9a6518d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:16.125650 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac5f9f-89ec-4339-8462-913e5af2fba4 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:16.127691 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858b7e6-4a57-4064-85cf-aae3f9a6518d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:16.131731 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a01079-2865-461f-9ef4-2c310661e320 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:16.132330 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a01079-2865-461f-9ef4-2c310661e320 service nova] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:16.132607 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a01079-2865-461f-9ef4-2c310661e320 service nova] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:16.132908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a01079-2865-461f-9ef4-2c310661e320 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:16.137774 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:16.141279 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a01079-2865-461f-9ef4-2c310661e320 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:16.142965 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a01079-2865-461f-9ef4-2c310661e320 service nova] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:16.153145 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1189/2371] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:43:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:43:16.160941 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1183/2372] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:43:16 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:43:16.162065 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a01079-2865-461f-9ef4-2c310661e320 service nova] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:16.172225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a01079-2865-461f-9ef4-2c310661e320 service nova] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:16.231492 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4da19d-76c5-45d0-9aef-41e119d6c687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:16.232139 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4da19d-76c5-45d0-9aef-41e119d6c687 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:16.232418 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4da19d-76c5-45d0-9aef-41e119d6c687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:16.232695 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4da19d-76c5-45d0-9aef-41e119d6c687 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:16.233374 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4da19d-76c5-45d0-9aef-41e119d6c687 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:16.235297 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4da19d-76c5-45d0-9aef-41e119d6c687 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:16.257756 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1190/2373] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:43:16 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:43:16.262306 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858b7e6-4a57-4064-85cf-aae3f9a6518d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:16.273210 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858b7e6-4a57-4064-85cf-aae3f9a6518d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:16.277796 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1ae8d-fb6f-48f6-93aa-acb50810e75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:16.278619 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1ae8d-fb6f-48f6-93aa-acb50810e75e admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:16.278898 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1ae8d-fb6f-48f6-93aa-acb50810e75e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:16.279165 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1ae8d-fb6f-48f6-93aa-acb50810e75e admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:16.279828 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1ae8d-fb6f-48f6-93aa-acb50810e75e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:16.281440 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1ae8d-fb6f-48f6-93aa-acb50810e75e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:16.292826 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1184/2374] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:43:16.297337 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4da19d-76c5-45d0-9aef-41e119d6c687 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:16.308103 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4da19d-76c5-45d0-9aef-41e119d6c687 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:16.313088 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e61ea-f04b-466d-82e4-0b91aa8b51ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:16.314467 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e61ea-f04b-466d-82e4-0b91aa8b51ca admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:16.314923 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e61ea-f04b-466d-82e4-0b91aa8b51ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:16.315211 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e61ea-f04b-466d-82e4-0b91aa8b51ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:16.315868 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e61ea-f04b-466d-82e4-0b91aa8b51ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:16.317694 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e61ea-f04b-466d-82e4-0b91aa8b51ca admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:16.329200 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1191/2375] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:43:16.334689 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1ae8d-fb6f-48f6-93aa-acb50810e75e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:16.345391 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1ae8d-fb6f-48f6-93aa-acb50810e75e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:16.350435 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749530df-744c-48cf-acc5-585760205bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:16.351080 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749530df-744c-48cf-acc5-585760205bed admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:16.351358 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749530df-744c-48cf-acc5-585760205bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:16.351630 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749530df-744c-48cf-acc5-585760205bed admin admin] PATH_INFO: `/v3/projects/d2f47c67111a49e184409ce751039485/users/c78507eb11ef44609970b6746c6638e9/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:16.367962 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749530df-744c-48cf-acc5-585760205bed admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2f47c67111a49e184409ce751039485, user_id=c78507eb11ef44609970b6746c6638e9, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:16.370628 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749530df-744c-48cf-acc5-585760205bed admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:16.385423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1185/2376] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:43:16 2022] PUT /identity/v3/projects/d2f47c67111a49e184409ce751039485/users/c78507eb11ef44609970b6746c6638e9/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:16.452907 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ec18e-9e42-4dd0-8582-7743d5df94b4 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:16.453204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ec18e-9e42-4dd0-8582-7743d5df94b4 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:16.453555 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ec18e-9e42-4dd0-8582-7743d5df94b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:16.490368 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-a62ec18e-9e42-4dd0-8582-7743d5df94b4 None None] MFA Rules not processed for user `c78507eb11ef44609970b6746c6638e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:16.492139 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-a62ec18e-9e42-4dd0-8582-7743d5df94b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:16.536826 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:16.580626 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1192/2377] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:43:16.592351 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749530df-744c-48cf-acc5-585760205bed admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:16.594024 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-749530df-744c-48cf-acc5-585760205bed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:16.650922 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749530df-744c-48cf-acc5-585760205bed admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:16.654586 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d891b00-d02d-4b3e-8250-a6c1b52abc7d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87b5093e963c4753928a8c6091afc52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:16.655185 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d891b00-d02d-4b3e-8250-a6c1b52abc7d service glance] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:16.655470 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d891b00-d02d-4b3e-8250-a6c1b52abc7d service glance] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:16.655747 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d891b00-d02d-4b3e-8250-a6c1b52abc7d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:16.664167 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d891b00-d02d-4b3e-8250-a6c1b52abc7d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:16.665618 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d891b00-d02d-4b3e-8250-a6c1b52abc7d service glance] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:16.674951 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1186/2378] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:43:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:43:18.881919 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ec18e-9e42-4dd0-8582-7743d5df94b4 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:18.883442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-a62ec18e-9e42-4dd0-8582-7743d5df94b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:18.950928 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ec18e-9e42-4dd0-8582-7743d5df94b4 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:18.956177 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898ed76-7a0e-439c-83a9-6f2ee8bc6e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:18.956853 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898ed76-7a0e-439c-83a9-6f2ee8bc6e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:18.957177 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898ed76-7a0e-439c-83a9-6f2ee8bc6e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:18.957453 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898ed76-7a0e-439c-83a9-6f2ee8bc6e7c admin admin] PATH_INFO: `/v3/users/67d31e578dbc4b3abd842d3a755b5467` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:18.958575 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898ed76-7a0e-439c-83a9-6f2ee8bc6e7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=67d31e578dbc4b3abd842d3a755b5467)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:18.960518 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898ed76-7a0e-439c-83a9-6f2ee8bc6e7c admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:19.006448 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-9898ed76-7a0e-439c-83a9-6f2ee8bc6e7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67d31e578dbc4b3abd842d3a755b5467'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:19.007015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-9898ed76-7a0e-439c-83a9-6f2ee8bc6e7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67d31e578dbc4b3abd842d3a755b5467'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:19.013554 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-9898ed76-7a0e-439c-83a9-6f2ee8bc6e7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67d31e578dbc4b3abd842d3a755b5467'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:19.014390 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-9898ed76-7a0e-439c-83a9-6f2ee8bc6e7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67d31e578dbc4b3abd842d3a755b5467'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:19.022298 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-9898ed76-7a0e-439c-83a9-6f2ee8bc6e7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67d31e578dbc4b3abd842d3a755b5467'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:19.040731 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1193/2379] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:43:18 2022] DELETE /identity/v3/users/67d31e578dbc4b3abd842d3a755b5467 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:19.045410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f63ce7-e35c-4997-81cb-b4ea9465e958 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:19.045917 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f63ce7-e35c-4997-81cb-b4ea9465e958 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:19.046051 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f63ce7-e35c-4997-81cb-b4ea9465e958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:19.066067 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-92f63ce7-e35c-4997-81cb-b4ea9465e958 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:19.067266 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-92f63ce7-e35c-4997-81cb-b4ea9465e958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:19.130511 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1187/2380] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:19.140111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898ed76-7a0e-439c-83a9-6f2ee8bc6e7c admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:19.141691 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-9898ed76-7a0e-439c-83a9-6f2ee8bc6e7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:19.226215 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898ed76-7a0e-439c-83a9-6f2ee8bc6e7c admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:19.230308 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186c492-313f-4fe5-80bb-c40b4d917f8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:19.231195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186c492-313f-4fe5-80bb-c40b4d917f8a service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:19.231564 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186c492-313f-4fe5-80bb-c40b4d917f8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:19.232289 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186c492-313f-4fe5-80bb-c40b4d917f8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:19.246624 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:19.249738 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186c492-313f-4fe5-80bb-c40b4d917f8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:19.253551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b186c492-313f-4fe5-80bb-c40b4d917f8a service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:19.269208 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1194/2381] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:43:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:43:19.571006 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f63ce7-e35c-4997-81cb-b4ea9465e958 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:19.572779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-92f63ce7-e35c-4997-81cb-b4ea9465e958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:19.590528 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f63ce7-e35c-4997-81cb-b4ea9465e958 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:19.596534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9f64e-81d5-4e77-a114-4dd22e73056c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:19.597287 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9f64e-81d5-4e77-a114-4dd22e73056c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:19.597638 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9f64e-81d5-4e77-a114-4dd22e73056c admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:19.597983 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9f64e-81d5-4e77-a114-4dd22e73056c admin admin] PATH_INFO: `/v3/projects/70556d4cdc8145fda2a9826761e9e20a` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:19.599639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9f64e-81d5-4e77-a114-4dd22e73056c admin admin] RBAC: Authorizing `identity:delete_project(project_id=70556d4cdc8145fda2a9826761e9e20a)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:19.601520 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9f64e-81d5-4e77-a114-4dd22e73056c admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:19.659630 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5cb9f64e-81d5-4e77-a114-4dd22e73056c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70556d4cdc8145fda2a9826761e9e20a', 'request_id': 'req-5cb9f64e-81d5-4e77-a114-4dd22e73056c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:19.661817 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:19.668289 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5cb9f64e-81d5-4e77-a114-4dd22e73056c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70556d4cdc8145fda2a9826761e9e20a', 'request_id': 'req-5cb9f64e-81d5-4e77-a114-4dd22e73056c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:19.678501 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5cb9f64e-81d5-4e77-a114-4dd22e73056c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70556d4cdc8145fda2a9826761e9e20a', 'request_id': 'req-5cb9f64e-81d5-4e77-a114-4dd22e73056c'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:19.692426 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5cb9f64e-81d5-4e77-a114-4dd22e73056c admin admin] The token cache is being invalidate because project 70556d4cdc8145fda2a9826761e9e20a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:43:19.692725 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5cb9f64e-81d5-4e77-a114-4dd22e73056c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:19.693727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5cb9f64e-81d5-4e77-a114-4dd22e73056c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:19.695658 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1188/2382] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:43:19 2022] DELETE /identity/v3/projects/70556d4cdc8145fda2a9826761e9e20a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:43:19.972397 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186c492-313f-4fe5-80bb-c40b4d917f8a service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:19.973986 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b186c492-313f-4fe5-80bb-c40b4d917f8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:19.989045 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b186c492-313f-4fe5-80bb-c40b4d917f8a service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:20.073060 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299282d-9afd-4aa8-ae2c-85af03704cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:20.073697 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299282d-9afd-4aa8-ae2c-85af03704cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:20.074037 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299282d-9afd-4aa8-ae2c-85af03704cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:20.074780 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299282d-9afd-4aa8-ae2c-85af03704cac admin admin] PATH_INFO: `/v3/users/704f8cb826214bc78dd49d0962f1417a` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:20.075722 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299282d-9afd-4aa8-ae2c-85af03704cac admin admin] RBAC: Authorizing `identity:delete_user(user_id=704f8cb826214bc78dd49d0962f1417a)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:20.077743 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299282d-9afd-4aa8-ae2c-85af03704cac admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:20.131638 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4299282d-9afd-4aa8-ae2c-85af03704cac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '704f8cb826214bc78dd49d0962f1417a'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:20.132389 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4299282d-9afd-4aa8-ae2c-85af03704cac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '704f8cb826214bc78dd49d0962f1417a'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:20.140339 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4299282d-9afd-4aa8-ae2c-85af03704cac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '704f8cb826214bc78dd49d0962f1417a'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:20.141045 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4299282d-9afd-4aa8-ae2c-85af03704cac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '704f8cb826214bc78dd49d0962f1417a'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:20.149893 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4299282d-9afd-4aa8-ae2c-85af03704cac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '704f8cb826214bc78dd49d0962f1417a'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:20.167579 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1195/2383] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:43:19 2022] DELETE /identity/v3/users/704f8cb826214bc78dd49d0962f1417a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:20.172000 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971691a-0f6f-4245-8cd2-6e9dff08df40 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:20.172473 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971691a-0f6f-4245-8cd2-6e9dff08df40 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:20.172569 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971691a-0f6f-4245-8cd2-6e9dff08df40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:20.199728 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2971691a-0f6f-4245-8cd2-6e9dff08df40 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:20.201136 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2971691a-0f6f-4245-8cd2-6e9dff08df40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:20.268760 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1189/2384] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:20.278541 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299282d-9afd-4aa8-ae2c-85af03704cac admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:20.279957 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-4299282d-9afd-4aa8-ae2c-85af03704cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:20.362453 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299282d-9afd-4aa8-ae2c-85af03704cac admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:20.366136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e6d94e-3507-42cf-b55f-f4f417e3ec10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:20.367007 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6d94e-3507-42cf-b55f-f4f417e3ec10 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:20.367287 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6d94e-3507-42cf-b55f-f4f417e3ec10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:20.367561 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6d94e-3507-42cf-b55f-f4f417e3ec10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:20.377476 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e6d94e-3507-42cf-b55f-f4f417e3ec10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:20.378923 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e6d94e-3507-42cf-b55f-f4f417e3ec10 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:20.388720 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1196/2385] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:43:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:43:20.891544 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971691a-0f6f-4245-8cd2-6e9dff08df40 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:20.893758 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2971691a-0f6f-4245-8cd2-6e9dff08df40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:20.907723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971691a-0f6f-4245-8cd2-6e9dff08df40 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:20.917636 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10234c33-2b0d-4521-9f32-798e9e869aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:20.918298 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10234c33-2b0d-4521-9f32-798e9e869aa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:20.918578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10234c33-2b0d-4521-9f32-798e9e869aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:20.918856 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10234c33-2b0d-4521-9f32-798e9e869aa4 admin admin] PATH_INFO: `/v3/projects/9b0a0dff64c64661a05db93ee8a8fd7d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:20.920200 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10234c33-2b0d-4521-9f32-798e9e869aa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b0a0dff64c64661a05db93ee8a8fd7d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:20.922335 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10234c33-2b0d-4521-9f32-798e9e869aa4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:20.970879 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-10234c33-2b0d-4521-9f32-798e9e869aa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b0a0dff64c64661a05db93ee8a8fd7d', 'request_id': 'req-10234c33-2b0d-4521-9f32-798e9e869aa4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:20.977827 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-10234c33-2b0d-4521-9f32-798e9e869aa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b0a0dff64c64661a05db93ee8a8fd7d', 'request_id': 'req-10234c33-2b0d-4521-9f32-798e9e869aa4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:20.986360 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-10234c33-2b0d-4521-9f32-798e9e869aa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b0a0dff64c64661a05db93ee8a8fd7d', 'request_id': 'req-10234c33-2b0d-4521-9f32-798e9e869aa4'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:20.997317 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-10234c33-2b0d-4521-9f32-798e9e869aa4 admin admin] The token cache is being invalidate because project 9b0a0dff64c64661a05db93ee8a8fd7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:43:20.997608 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-10234c33-2b0d-4521-9f32-798e9e869aa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:20.998238 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-10234c33-2b0d-4521-9f32-798e9e869aa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:20.999754 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1190/2386] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:43:20 2022] DELETE /identity/v3/projects/9b0a0dff64c64661a05db93ee8a8fd7d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:21.005837 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a735a5-65b9-49db-be2c-4278a8d31676 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:21.006316 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a735a5-65b9-49db-be2c-4278a8d31676 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:21.006413 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a735a5-65b9-49db-be2c-4278a8d31676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:21.025953 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-76a735a5-65b9-49db-be2c-4278a8d31676 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:21.027145 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-76a735a5-65b9-49db-be2c-4278a8d31676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:21.036853 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:21.092590 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1197/2387] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:21.101751 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9dbb2b-9c52-48d0-a98c-af8f0db52ebc None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:21.102082 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9dbb2b-9c52-48d0-a98c-af8f0db52ebc None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:21.102386 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9dbb2b-9c52-48d0-a98c-af8f0db52ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:21.123714 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-4f9dbb2b-9c52-48d0-a98c-af8f0db52ebc None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:21.124973 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-4f9dbb2b-9c52-48d0-a98c-af8f0db52ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:21.133553 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1191/2388] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:21.138028 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a735a5-65b9-49db-be2c-4278a8d31676 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:21.147949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a735a5-65b9-49db-be2c-4278a8d31676 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:21.152510 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8d79d-90d2-444c-90c7-0f9b94e9f82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:21.153132 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8d79d-90d2-444c-90c7-0f9b94e9f82a admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:21.153414 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8d79d-90d2-444c-90c7-0f9b94e9f82a admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:21.153686 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8d79d-90d2-444c-90c7-0f9b94e9f82a admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:21.154364 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8d79d-90d2-444c-90c7-0f9b94e9f82a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:43:21.154665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8d79d-90d2-444c-90c7-0f9b94e9f82a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:21.156258 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8d79d-90d2-444c-90c7-0f9b94e9f82a admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:21.168155 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1198/2389] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:43:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:43:21.172340 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9dbb2b-9c52-48d0-a98c-af8f0db52ebc None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:21.182858 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9dbb2b-9c52-48d0-a98c-af8f0db52ebc None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:21.188097 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73cbd2-0920-4572-92ad-2559e38f59fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:21.189821 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73cbd2-0920-4572-92ad-2559e38f59fb admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:21.189821 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73cbd2-0920-4572-92ad-2559e38f59fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:21.190121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73cbd2-0920-4572-92ad-2559e38f59fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:21.190675 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73cbd2-0920-4572-92ad-2559e38f59fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:21.192611 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73cbd2-0920-4572-92ad-2559e38f59fb admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:21.219785 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1192/2390] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:43:21 2022] POST /identity/v3/projects => generated 393 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:43:21.224096 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8d79d-90d2-444c-90c7-0f9b94e9f82a admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:21.234007 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8d79d-90d2-444c-90c7-0f9b94e9f82a admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:21.291908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97e99b-a80c-4512-8971-fd87cad9e802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:21.292516 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97e99b-a80c-4512-8971-fd87cad9e802 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:21.292796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97e99b-a80c-4512-8971-fd87cad9e802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:21.293201 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97e99b-a80c-4512-8971-fd87cad9e802 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:21.293813 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97e99b-a80c-4512-8971-fd87cad9e802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:21.295758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97e99b-a80c-4512-8971-fd87cad9e802 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:21.318444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1199/2391] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:43:21 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:43:21.323944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73cbd2-0920-4572-92ad-2559e38f59fb admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:21.333819 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73cbd2-0920-4572-92ad-2559e38f59fb admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:21.338589 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa795365-b793-4b82-b3de-efd0eaa0442a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:21.339193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa795365-b793-4b82-b3de-efd0eaa0442a admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:21.339468 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa795365-b793-4b82-b3de-efd0eaa0442a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:21.339742 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa795365-b793-4b82-b3de-efd0eaa0442a admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:21.340379 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa795365-b793-4b82-b3de-efd0eaa0442a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:21.342005 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa795365-b793-4b82-b3de-efd0eaa0442a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:21.355193 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1193/2392] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:43:21.356462 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97e99b-a80c-4512-8971-fd87cad9e802 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:21.367374 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97e99b-a80c-4512-8971-fd87cad9e802 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:21.371934 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6431b-0718-472b-ab07-82a702ad0765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:21.372531 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6431b-0718-472b-ab07-82a702ad0765 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:21.372828 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6431b-0718-472b-ab07-82a702ad0765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:21.373119 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6431b-0718-472b-ab07-82a702ad0765 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:21.373841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6431b-0718-472b-ab07-82a702ad0765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:21.375583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6431b-0718-472b-ab07-82a702ad0765 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:21.387778 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1200/2393] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:43:21.391585 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa795365-b793-4b82-b3de-efd0eaa0442a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:21.401993 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa795365-b793-4b82-b3de-efd0eaa0442a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:21.406709 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a64471-44f5-4b2d-9c9f-6b2dad80d385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:21.407964 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a64471-44f5-4b2d-9c9f-6b2dad80d385 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:21.408302 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a64471-44f5-4b2d-9c9f-6b2dad80d385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:21.408576 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a64471-44f5-4b2d-9c9f-6b2dad80d385 admin admin] PATH_INFO: `/v3/projects/b0f1f41bf23440819c6b287a84e8ab27/users/09c806dc4bca4ae49814b24ee1a07cb5/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:21.424456 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a64471-44f5-4b2d-9c9f-6b2dad80d385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f1f41bf23440819c6b287a84e8ab27, user_id=09c806dc4bca4ae49814b24ee1a07cb5, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:21.427164 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a64471-44f5-4b2d-9c9f-6b2dad80d385 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:21.437400 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:21.443100 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1194/2394] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:43:21 2022] PUT /identity/v3/projects/b0f1f41bf23440819c6b287a84e8ab27/users/09c806dc4bca4ae49814b24ee1a07cb5/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:21.455054 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ede28-213b-4731-a998-59e6b9fa9481 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:21.455342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ede28-213b-4731-a998-59e6b9fa9481 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:21.455621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ede28-213b-4731-a998-59e6b9fa9481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:21.486594 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-416ede28-213b-4731-a998-59e6b9fa9481 None None] MFA Rules not processed for user `09c806dc4bca4ae49814b24ee1a07cb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:21.487814 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-416ede28-213b-4731-a998-59e6b9fa9481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:21.566006 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1201/2395] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:21.570955 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a64471-44f5-4b2d-9c9f-6b2dad80d385 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:21.633396 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a64471-44f5-4b2d-9c9f-6b2dad80d385 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:21.638589 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8070d-0242-4307-b2fc-00c87d5c13bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:21.639207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8070d-0242-4307-b2fc-00c87d5c13bb admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:21.639516 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8070d-0242-4307-b2fc-00c87d5c13bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:21.639818 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8070d-0242-4307-b2fc-00c87d5c13bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:21.640476 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8070d-0242-4307-b2fc-00c87d5c13bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:21.642486 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8070d-0242-4307-b2fc-00c87d5c13bb admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:21.671770 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1195/2396] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:43:21 2022] POST /identity/v3/projects => generated 391 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:43:21.675399 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ede28-213b-4731-a998-59e6b9fa9481 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:21.686035 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ede28-213b-4731-a998-59e6b9fa9481 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:21.697427 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:21.749907 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4785c-f7a8-4c42-864b-4d4290d4ca7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:21.750506 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4785c-f7a8-4c42-864b-4d4290d4ca7b admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:21.750784 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4785c-f7a8-4c42-864b-4d4290d4ca7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:21.751473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4785c-f7a8-4c42-864b-4d4290d4ca7b admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:21.751883 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f4785c-f7a8-4c42-864b-4d4290d4ca7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:21.753839 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f4785c-f7a8-4c42-864b-4d4290d4ca7b admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:21.778395 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1202/2397] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:43:21 2022] POST /identity/v3/users => generated 302 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:43:21.782872 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8070d-0242-4307-b2fc-00c87d5c13bb admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:21.793560 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8070d-0242-4307-b2fc-00c87d5c13bb admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:21.798192 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f4ca2-4f67-40ff-a0dd-25e4783c2cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:21.798785 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f4ca2-4f67-40ff-a0dd-25e4783c2cfe admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:21.799066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f4ca2-4f67-40ff-a0dd-25e4783c2cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:21.799348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f4ca2-4f67-40ff-a0dd-25e4783c2cfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:21.799989 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f4ca2-4f67-40ff-a0dd-25e4783c2cfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:21.801656 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f4ca2-4f67-40ff-a0dd-25e4783c2cfe admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:21.812021 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1196/2398] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:43:21.816099 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4785c-f7a8-4c42-864b-4d4290d4ca7b admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:21.826732 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4785c-f7a8-4c42-864b-4d4290d4ca7b admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:21.832508 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f0966-2d12-4eaa-a657-6c88399b983b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:21.833229 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f0966-2d12-4eaa-a657-6c88399b983b admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:21.833535 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f0966-2d12-4eaa-a657-6c88399b983b admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:21.833813 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f0966-2d12-4eaa-a657-6c88399b983b admin admin] PATH_INFO: `/v3/projects/645beed24f014d7d800c07d03033d8bb/users/63c712544c9640a9ba79383a019e9719/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:21.849719 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f0966-2d12-4eaa-a657-6c88399b983b admin admin] RBAC: Authorizing `identity:create_grant(project_id=645beed24f014d7d800c07d03033d8bb, user_id=63c712544c9640a9ba79383a019e9719, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:21.852361 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f0966-2d12-4eaa-a657-6c88399b983b admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:21.865176 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1203/2399] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:43:21 2022] PUT /identity/v3/projects/645beed24f014d7d800c07d03033d8bb/users/63c712544c9640a9ba79383a019e9719/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:21.869495 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f4ca2-4f67-40ff-a0dd-25e4783c2cfe admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:21.879494 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f4ca2-4f67-40ff-a0dd-25e4783c2cfe admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:21.937947 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9fb88-dc51-4ff5-bb3b-dbc99cd610d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:21.938520 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9fb88-dc51-4ff5-bb3b-dbc99cd610d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:21.939040 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9fb88-dc51-4ff5-bb3b-dbc99cd610d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:21.939040 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9fb88-dc51-4ff5-bb3b-dbc99cd610d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:21.939820 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad9fb88-dc51-4ff5-bb3b-dbc99cd610d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:21.941284 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad9fb88-dc51-4ff5-bb3b-dbc99cd610d1 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:21.951027 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1197/2400] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:43:21.956016 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f0966-2d12-4eaa-a657-6c88399b983b admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:21.967060 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f0966-2d12-4eaa-a657-6c88399b983b admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:21.972732 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8e983-b1af-484f-9b88-0f4521ef3dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:21.973435 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8e983-b1af-484f-9b88-0f4521ef3dee admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:21.973714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8e983-b1af-484f-9b88-0f4521ef3dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:21.974017 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8e983-b1af-484f-9b88-0f4521ef3dee admin admin] PATH_INFO: `/v3/projects/645beed24f014d7d800c07d03033d8bb/users/63c712544c9640a9ba79383a019e9719/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:21.975903 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8e983-b1af-484f-9b88-0f4521ef3dee admin admin] RBAC: Authorizing `identity:create_grant(project_id=645beed24f014d7d800c07d03033d8bb, user_id=63c712544c9640a9ba79383a019e9719, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:21.978579 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8e983-b1af-484f-9b88-0f4521ef3dee admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:21.988991 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1204/2401] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:43:21 2022] PUT /identity/v3/projects/645beed24f014d7d800c07d03033d8bb/users/63c712544c9640a9ba79383a019e9719/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:21.999425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154def5d-0aec-4fe2-9d31-5c96705219a7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:21.999717 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154def5d-0aec-4fe2-9d31-5c96705219a7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:21.999994 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154def5d-0aec-4fe2-9d31-5c96705219a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:22.029045 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-154def5d-0aec-4fe2-9d31-5c96705219a7 None None] MFA Rules not processed for user `63c712544c9640a9ba79383a019e9719`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:22.030414 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-154def5d-0aec-4fe2-9d31-5c96705219a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:22.094688 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:22.106121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1198/2402] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:21 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:22.117397 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8e983-b1af-484f-9b88-0f4521ef3dee admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:22.119204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-72b8e983-b1af-484f-9b88-0f4521ef3dee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:22.181165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8e983-b1af-484f-9b88-0f4521ef3dee admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:22.184748 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b968b-d836-4217-a639-5d3d8e000efe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:22.185376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b968b-d836-4217-a639-5d3d8e000efe service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:22.185656 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b968b-d836-4217-a639-5d3d8e000efe service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:22.185925 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b968b-d836-4217-a639-5d3d8e000efe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:22.195926 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b968b-d836-4217-a639-5d3d8e000efe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:22.197333 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b968b-d836-4217-a639-5d3d8e000efe service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:22.207310 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1205/2403] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:43:22 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:43:22.256104 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154def5d-0aec-4fe2-9d31-5c96705219a7 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:22.257620 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-154def5d-0aec-4fe2-9d31-5c96705219a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:22.271095 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154def5d-0aec-4fe2-9d31-5c96705219a7 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:22.274657 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec10411-f400-4947-950b-88d35aa0e1a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:22.275284 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec10411-f400-4947-950b-88d35aa0e1a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:22.275566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec10411-f400-4947-950b-88d35aa0e1a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:22.275833 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec10411-f400-4947-950b-88d35aa0e1a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:22.333303 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec10411-f400-4947-950b-88d35aa0e1a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:22.334756 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec10411-f400-4947-950b-88d35aa0e1a6 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:22.343641 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1199/2404] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:43:22 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:43:23.771125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439095ff-56fa-4bb2-9803-fc9b2134f069 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:23.771662 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439095ff-56fa-4bb2-9803-fc9b2134f069 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:23.772171 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439095ff-56fa-4bb2-9803-fc9b2134f069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:23.792769 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-439095ff-56fa-4bb2-9803-fc9b2134f069 None None] MFA Rules not processed for user `85bb137d0fb74031a49a5c629b32c32b`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:23.794001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-439095ff-56fa-4bb2-9803-fc9b2134f069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:23.855251 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1206/2405] 10.222.0.23 () {58 vars in 967 bytes} [Thu Oct 20 13:43:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:23.863723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec10411-f400-4947-950b-88d35aa0e1a6 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:23.865149 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-aec10411-f400-4947-950b-88d35aa0e1a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:23.936034 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec10411-f400-4947-950b-88d35aa0e1a6 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:23.941885 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffc1cb-ba03-4690-b34c-cf9a7bf93a35 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9834211488354651803000c7e8fab80b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:23.942497 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffc1cb-ba03-4690-b34c-cf9a7bf93a35 service swift] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:23.942776 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffc1cb-ba03-4690-b34c-cf9a7bf93a35 service swift] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:23.943051 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffc1cb-ba03-4690-b34c-cf9a7bf93a35 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:23.951055 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ffc1cb-ba03-4690-b34c-cf9a7bf93a35 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:23.952626 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ffc1cb-ba03-4690-b34c-cf9a7bf93a35 service swift] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:23.961414 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1200/2406] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 20 13:43:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 13:43:26.955588 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439095ff-56fa-4bb2-9803-fc9b2134f069 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:26.971956 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:27.036251 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439095ff-56fa-4bb2-9803-fc9b2134f069 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:27.040858 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e0173-f9df-4f60-af52-d1466d80939e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:27.041508 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e0173-f9df-4f60-af52-d1466d80939e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:27.041783 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e0173-f9df-4f60-af52-d1466d80939e admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:27.042055 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e0173-f9df-4f60-af52-d1466d80939e admin admin] PATH_INFO: `/v3/users/09c806dc4bca4ae49814b24ee1a07cb5` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:27.043177 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e0173-f9df-4f60-af52-d1466d80939e admin admin] RBAC: Authorizing `identity:delete_user(user_id=09c806dc4bca4ae49814b24ee1a07cb5)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:27.045132 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e0173-f9df-4f60-af52-d1466d80939e admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:27.093955 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-418e0173-f9df-4f60-af52-d1466d80939e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09c806dc4bca4ae49814b24ee1a07cb5'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:27.095273 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-418e0173-f9df-4f60-af52-d1466d80939e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09c806dc4bca4ae49814b24ee1a07cb5'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:27.103320 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-418e0173-f9df-4f60-af52-d1466d80939e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09c806dc4bca4ae49814b24ee1a07cb5'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:27.103996 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-418e0173-f9df-4f60-af52-d1466d80939e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09c806dc4bca4ae49814b24ee1a07cb5'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:27.111765 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-418e0173-f9df-4f60-af52-d1466d80939e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09c806dc4bca4ae49814b24ee1a07cb5'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:27.131575 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1207/2407] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:43:26 2022] DELETE /identity/v3/users/09c806dc4bca4ae49814b24ee1a07cb5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:27.136383 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d604516-0323-4546-bbd1-244bd743911b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:27.136912 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d604516-0323-4546-bbd1-244bd743911b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:27.137264 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d604516-0323-4546-bbd1-244bd743911b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:27.157905 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2d604516-0323-4546-bbd1-244bd743911b None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:27.176197 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2d604516-0323-4546-bbd1-244bd743911b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:27.253306 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1201/2408] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:27.263270 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e0173-f9df-4f60-af52-d1466d80939e admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:27.264693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-418e0173-f9df-4f60-af52-d1466d80939e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:27.339579 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e0173-f9df-4f60-af52-d1466d80939e admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:27.344952 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33d5ae-ba44-4b90-b8d5-0d476c8c636a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:27.345595 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33d5ae-ba44-4b90-b8d5-0d476c8c636a service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:27.345905 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33d5ae-ba44-4b90-b8d5-0d476c8c636a service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:27.346153 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33d5ae-ba44-4b90-b8d5-0d476c8c636a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:27.362620 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc33d5ae-ba44-4b90-b8d5-0d476c8c636a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:27.364078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc33d5ae-ba44-4b90-b8d5-0d476c8c636a service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:27.378670 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1208/2409] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:43:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:43:28.079945 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d604516-0323-4546-bbd1-244bd743911b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:28.082371 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2d604516-0323-4546-bbd1-244bd743911b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:28.098891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d604516-0323-4546-bbd1-244bd743911b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:28.105135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b518d3-defd-4078-82fd-a54525f6eab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:28.108463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b518d3-defd-4078-82fd-a54525f6eab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:28.108747 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b518d3-defd-4078-82fd-a54525f6eab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:28.109230 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b518d3-defd-4078-82fd-a54525f6eab9 admin admin] PATH_INFO: `/v3/projects/b0f1f41bf23440819c6b287a84e8ab27` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:28.112037 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b518d3-defd-4078-82fd-a54525f6eab9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0f1f41bf23440819c6b287a84e8ab27)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:28.115153 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b518d3-defd-4078-82fd-a54525f6eab9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:28.140721 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:28.168850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-01b518d3-defd-4078-82fd-a54525f6eab9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0f1f41bf23440819c6b287a84e8ab27', 'request_id': 'req-01b518d3-defd-4078-82fd-a54525f6eab9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:28.178687 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-01b518d3-defd-4078-82fd-a54525f6eab9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0f1f41bf23440819c6b287a84e8ab27', 'request_id': 'req-01b518d3-defd-4078-82fd-a54525f6eab9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:28.191223 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-01b518d3-defd-4078-82fd-a54525f6eab9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0f1f41bf23440819c6b287a84e8ab27', 'request_id': 'req-01b518d3-defd-4078-82fd-a54525f6eab9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:28.204349 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-01b518d3-defd-4078-82fd-a54525f6eab9 admin admin] The token cache is being invalidate because project b0f1f41bf23440819c6b287a84e8ab27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:43:28.204641 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-01b518d3-defd-4078-82fd-a54525f6eab9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:28.205340 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-01b518d3-defd-4078-82fd-a54525f6eab9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:28.206858 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1202/2410] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:43:28 2022] DELETE /identity/v3/projects/b0f1f41bf23440819c6b287a84e8ab27 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:28.210872 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33d5ae-ba44-4b90-b8d5-0d476c8c636a service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:28.221024 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33d5ae-ba44-4b90-b8d5-0d476c8c636a service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:28.254844 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:28.286090 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d80540-c087-42fe-8f3c-9ff44f067160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:28.287177 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d80540-c087-42fe-8f3c-9ff44f067160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:28.288008 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d80540-c087-42fe-8f3c-9ff44f067160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:28.288207 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d80540-c087-42fe-8f3c-9ff44f067160 admin admin] PATH_INFO: `/v3/users/63c712544c9640a9ba79383a019e9719` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:28.289465 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d80540-c087-42fe-8f3c-9ff44f067160 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63c712544c9640a9ba79383a019e9719)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:28.291422 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d80540-c087-42fe-8f3c-9ff44f067160 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:28.342619 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-43d80540-c087-42fe-8f3c-9ff44f067160 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63c712544c9640a9ba79383a019e9719'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:28.343558 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-43d80540-c087-42fe-8f3c-9ff44f067160 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63c712544c9640a9ba79383a019e9719'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:28.352997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-43d80540-c087-42fe-8f3c-9ff44f067160 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63c712544c9640a9ba79383a019e9719'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:28.353758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-43d80540-c087-42fe-8f3c-9ff44f067160 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63c712544c9640a9ba79383a019e9719'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:28.362847 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-43d80540-c087-42fe-8f3c-9ff44f067160 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63c712544c9640a9ba79383a019e9719'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:28.381460 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1209/2411] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:43:28 2022] DELETE /identity/v3/users/63c712544c9640a9ba79383a019e9719 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:43:28.674697 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b518d3-defd-4078-82fd-a54525f6eab9 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:28.676188 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-01b518d3-defd-4078-82fd-a54525f6eab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:28.691917 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b518d3-defd-4078-82fd-a54525f6eab9 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:28.760853 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b1924-9414-4489-be93-1a5809fb08b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:28.761408 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b1924-9414-4489-be93-1a5809fb08b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:28.761688 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b1924-9414-4489-be93-1a5809fb08b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:28.761961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b1924-9414-4489-be93-1a5809fb08b9 admin admin] PATH_INFO: `/v3/projects/645beed24f014d7d800c07d03033d8bb` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:28.763744 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b1924-9414-4489-be93-1a5809fb08b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=645beed24f014d7d800c07d03033d8bb)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:28.765656 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b1924-9414-4489-be93-1a5809fb08b9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:28.827795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-371b1924-9414-4489-be93-1a5809fb08b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '645beed24f014d7d800c07d03033d8bb', 'request_id': 'req-371b1924-9414-4489-be93-1a5809fb08b9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:28.836015 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-371b1924-9414-4489-be93-1a5809fb08b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '645beed24f014d7d800c07d03033d8bb', 'request_id': 'req-371b1924-9414-4489-be93-1a5809fb08b9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:28.849344 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-371b1924-9414-4489-be93-1a5809fb08b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '645beed24f014d7d800c07d03033d8bb', 'request_id': 'req-371b1924-9414-4489-be93-1a5809fb08b9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:28.863533 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-371b1924-9414-4489-be93-1a5809fb08b9 admin admin] The token cache is being invalidate because project 645beed24f014d7d800c07d03033d8bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:43:28.864133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-371b1924-9414-4489-be93-1a5809fb08b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:28.864954 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-371b1924-9414-4489-be93-1a5809fb08b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:28.867430 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1203/2412] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:43:28 2022] DELETE /identity/v3/projects/645beed24f014d7d800c07d03033d8bb => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:43:28.873131 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d38e0-3485-475b-ad0f-7795eb818188 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:28.873421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d38e0-3485-475b-ad0f-7795eb818188 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:28.873700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d38e0-3485-475b-ad0f-7795eb818188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:28.897167 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-a21d38e0-3485-475b-ad0f-7795eb818188 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:28.898444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-a21d38e0-3485-475b-ad0f-7795eb818188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:28.971637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1210/2413] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:28.981190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf13f5-566d-4837-80e7-a9752c4c24ce None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:28.981501 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf13f5-566d-4837-80e7-a9752c4c24ce None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:28.981749 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf13f5-566d-4837-80e7-a9752c4c24ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:29.006796 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-28cf13f5-566d-4837-80e7-a9752c4c24ce None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:29.008734 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-28cf13f5-566d-4837-80e7-a9752c4c24ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:29.018221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1204/2414] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:29.022617 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d38e0-3485-475b-ad0f-7795eb818188 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:29.033765 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d38e0-3485-475b-ad0f-7795eb818188 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:29.038434 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a0a4f-bc92-4f33-b1b9-aeba2a5fd356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:29.039158 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a0a4f-bc92-4f33-b1b9-aeba2a5fd356 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:29.039376 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a0a4f-bc92-4f33-b1b9-aeba2a5fd356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:29.039572 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a0a4f-bc92-4f33-b1b9-aeba2a5fd356 admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:29.040231 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a0a4f-bc92-4f33-b1b9-aeba2a5fd356 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:43:29.040566 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a0a4f-bc92-4f33-b1b9-aeba2a5fd356 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:29.042220 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a0a4f-bc92-4f33-b1b9-aeba2a5fd356 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:29.057234 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1211/2415] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:43:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:43:29.061404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf13f5-566d-4837-80e7-a9752c4c24ce None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:29.071898 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf13f5-566d-4837-80e7-a9752c4c24ce None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:29.076704 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1d1ce-bc57-42e6-8c4d-b06b98a0a127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:29.077344 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1d1ce-bc57-42e6-8c4d-b06b98a0a127 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:29.077624 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1d1ce-bc57-42e6-8c4d-b06b98a0a127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:29.077898 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1d1ce-bc57-42e6-8c4d-b06b98a0a127 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:29.078544 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1d1ce-bc57-42e6-8c4d-b06b98a0a127 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:29.081199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1d1ce-bc57-42e6-8c4d-b06b98a0a127 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:29.111741 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1205/2416] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:43:29 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:43:29.115856 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a0a4f-bc92-4f33-b1b9-aeba2a5fd356 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:29.127122 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a0a4f-bc92-4f33-b1b9-aeba2a5fd356 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:29.168684 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:29.212688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88837b7b-58f6-46a4-8c7e-2a5bc8df18ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:29.214450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88837b7b-58f6-46a4-8c7e-2a5bc8df18ac admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:29.215083 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88837b7b-58f6-46a4-8c7e-2a5bc8df18ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:29.216193 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88837b7b-58f6-46a4-8c7e-2a5bc8df18ac admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:29.217165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88837b7b-58f6-46a4-8c7e-2a5bc8df18ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:29.220298 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88837b7b-58f6-46a4-8c7e-2a5bc8df18ac admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:29.252140 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1212/2417] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:43:29 2022] POST /identity/v3/users => generated 289 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:43:29.253359 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1d1ce-bc57-42e6-8c4d-b06b98a0a127 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:29.264107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1d1ce-bc57-42e6-8c4d-b06b98a0a127 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:29.272611 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff3114d-8cb8-4df6-9983-b493503d3959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:29.273818 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3114d-8cb8-4df6-9983-b493503d3959 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:29.274107 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3114d-8cb8-4df6-9983-b493503d3959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:29.274381 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3114d-8cb8-4df6-9983-b493503d3959 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:29.275082 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff3114d-8cb8-4df6-9983-b493503d3959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:29.276782 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff3114d-8cb8-4df6-9983-b493503d3959 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:29.291477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:29.297823 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1206/2418] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:43:29.302159 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88837b7b-58f6-46a4-8c7e-2a5bc8df18ac admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:29.316276 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88837b7b-58f6-46a4-8c7e-2a5bc8df18ac admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:29.321635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5795dfa5-2bdd-4cea-8ab5-51cd1153be60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:29.322445 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795dfa5-2bdd-4cea-8ab5-51cd1153be60 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:29.322940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795dfa5-2bdd-4cea-8ab5-51cd1153be60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:29.323295 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795dfa5-2bdd-4cea-8ab5-51cd1153be60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:29.324024 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5795dfa5-2bdd-4cea-8ab5-51cd1153be60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:29.325762 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5795dfa5-2bdd-4cea-8ab5-51cd1153be60 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:29.338086 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1213/2419] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:43:29.342380 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff3114d-8cb8-4df6-9983-b493503d3959 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:29.354017 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff3114d-8cb8-4df6-9983-b493503d3959 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:29.365811 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fe075-691c-42ea-af27-c3acc58ac0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:29.366955 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fe075-691c-42ea-af27-c3acc58ac0cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:29.367409 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fe075-691c-42ea-af27-c3acc58ac0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:29.367829 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fe075-691c-42ea-af27-c3acc58ac0cf admin admin] PATH_INFO: `/v3/projects/b947d0325b7245dda895da87fc8a28ec/users/a1a0bc0d72614fabba6d24fa88e7d8bd/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:29.415500 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fe075-691c-42ea-af27-c3acc58ac0cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b947d0325b7245dda895da87fc8a28ec, user_id=a1a0bc0d72614fabba6d24fa88e7d8bd, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:29.418529 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fe075-691c-42ea-af27-c3acc58ac0cf admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:29.442180 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1207/2420] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:43:29 2022] PUT /identity/v3/projects/b947d0325b7245dda895da87fc8a28ec/users/a1a0bc0d72614fabba6d24fa88e7d8bd/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:29.452705 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cba946-8cd0-42a9-8e36-25ed360624a0 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:29.453441 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cba946-8cd0-42a9-8e36-25ed360624a0 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:29.454242 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cba946-8cd0-42a9-8e36-25ed360624a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:29.493828 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-40cba946-8cd0-42a9-8e36-25ed360624a0 None None] MFA Rules not processed for user `a1a0bc0d72614fabba6d24fa88e7d8bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:29.495513 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-40cba946-8cd0-42a9-8e36-25ed360624a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:29.609506 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1214/2421] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:29.620641 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fe075-691c-42ea-af27-c3acc58ac0cf admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:29.624097 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-656fe075-691c-42ea-af27-c3acc58ac0cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:29.692967 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fe075-691c-42ea-af27-c3acc58ac0cf admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:29.701031 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0a429-f747-4918-a501-f6b808703a11 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87b5093e963c4753928a8c6091afc52b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:29.701870 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0a429-f747-4918-a501-f6b808703a11 service glance] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:29.702157 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0a429-f747-4918-a501-f6b808703a11 service glance] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:29.702431 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0a429-f747-4918-a501-f6b808703a11 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:29.712068 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a0a429-f747-4918-a501-f6b808703a11 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:29.713497 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a0a429-f747-4918-a501-f6b808703a11 service glance] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:29.724524 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1208/2422] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:43:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:43:31.459529 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cba946-8cd0-42a9-8e36-25ed360624a0 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:31.463691 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-40cba946-8cd0-42a9-8e36-25ed360624a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:31.610242 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:31.623091 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cba946-8cd0-42a9-8e36-25ed360624a0 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:31.628233 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb8863-787b-48d0-8c44-fe41af837eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:31.628901 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb8863-787b-48d0-8c44-fe41af837eee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:31.629173 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb8863-787b-48d0-8c44-fe41af837eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:31.629452 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb8863-787b-48d0-8c44-fe41af837eee admin admin] PATH_INFO: `/v3/users/ce7ce255f03e4d999ff111ccf3a799d7` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:31.630648 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb8863-787b-48d0-8c44-fe41af837eee admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce7ce255f03e4d999ff111ccf3a799d7)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:31.632592 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb8863-787b-48d0-8c44-fe41af837eee admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:31.678874 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4afb8863-787b-48d0-8c44-fe41af837eee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce7ce255f03e4d999ff111ccf3a799d7'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:31.679518 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4afb8863-787b-48d0-8c44-fe41af837eee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce7ce255f03e4d999ff111ccf3a799d7'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:31.686702 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4afb8863-787b-48d0-8c44-fe41af837eee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce7ce255f03e4d999ff111ccf3a799d7'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:31.687310 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4afb8863-787b-48d0-8c44-fe41af837eee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce7ce255f03e4d999ff111ccf3a799d7'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:31.695093 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4afb8863-787b-48d0-8c44-fe41af837eee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce7ce255f03e4d999ff111ccf3a799d7'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:31.712217 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1215/2423] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:43:31 2022] DELETE /identity/v3/users/ce7ce255f03e4d999ff111ccf3a799d7 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:31.893934 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0a429-f747-4918-a501-f6b808703a11 service glance] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:31.925575 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0a429-f747-4918-a501-f6b808703a11 service glance] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:32.005525 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce4811-2764-4192-9e5c-db7fe3d7c06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:32.006566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce4811-2764-4192-9e5c-db7fe3d7c06d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:32.007406 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce4811-2764-4192-9e5c-db7fe3d7c06d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:32.007688 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce4811-2764-4192-9e5c-db7fe3d7c06d admin admin] PATH_INFO: `/v3/projects/accdb851c01342d49228faa016ff430c` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:32.008984 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce4811-2764-4192-9e5c-db7fe3d7c06d admin admin] RBAC: Authorizing `identity:delete_project(project_id=accdb851c01342d49228faa016ff430c)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:32.011041 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce4811-2764-4192-9e5c-db7fe3d7c06d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:32.071163 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-abce4811-2764-4192-9e5c-db7fe3d7c06d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'accdb851c01342d49228faa016ff430c', 'request_id': 'req-abce4811-2764-4192-9e5c-db7fe3d7c06d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:32.078319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-abce4811-2764-4192-9e5c-db7fe3d7c06d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'accdb851c01342d49228faa016ff430c', 'request_id': 'req-abce4811-2764-4192-9e5c-db7fe3d7c06d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:32.087272 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-abce4811-2764-4192-9e5c-db7fe3d7c06d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'accdb851c01342d49228faa016ff430c', 'request_id': 'req-abce4811-2764-4192-9e5c-db7fe3d7c06d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:32.088720 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:32.101227 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-abce4811-2764-4192-9e5c-db7fe3d7c06d admin admin] The token cache is being invalidate because project accdb851c01342d49228faa016ff430c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:43:32.101520 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-abce4811-2764-4192-9e5c-db7fe3d7c06d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:32.102173 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-abce4811-2764-4192-9e5c-db7fe3d7c06d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:32.103945 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1209/2424] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:43:31 2022] DELETE /identity/v3/projects/accdb851c01342d49228faa016ff430c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:32.108070 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb8863-787b-48d0-8c44-fe41af837eee admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:32.110856 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-4afb8863-787b-48d0-8c44-fe41af837eee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:32.124215 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb8863-787b-48d0-8c44-fe41af837eee admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:32.181756 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7da96e-5064-4d09-8515-79103869e569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:32.182400 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7da96e-5064-4d09-8515-79103869e569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:32.182683 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7da96e-5064-4d09-8515-79103869e569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:32.182957 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7da96e-5064-4d09-8515-79103869e569 admin admin] PATH_INFO: `/v3/users/81b411dd56004e84abb03889b165b2ba` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:32.184141 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7da96e-5064-4d09-8515-79103869e569 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81b411dd56004e84abb03889b165b2ba)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:32.186064 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7da96e-5064-4d09-8515-79103869e569 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:32.235971 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ed7da96e-5064-4d09-8515-79103869e569 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81b411dd56004e84abb03889b165b2ba'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:32.236622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ed7da96e-5064-4d09-8515-79103869e569 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81b411dd56004e84abb03889b165b2ba'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:32.243999 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ed7da96e-5064-4d09-8515-79103869e569 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81b411dd56004e84abb03889b165b2ba'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:32.244741 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ed7da96e-5064-4d09-8515-79103869e569 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81b411dd56004e84abb03889b165b2ba'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:32.252649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-ed7da96e-5064-4d09-8515-79103869e569 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81b411dd56004e84abb03889b165b2ba'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:32.259961 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:32.275038 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1216/2425] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:43:32 2022] DELETE /identity/v3/users/81b411dd56004e84abb03889b165b2ba => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:32.447600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce4811-2764-4192-9e5c-db7fe3d7c06d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:32.449102 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-abce4811-2764-4192-9e5c-db7fe3d7c06d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:32.464278 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce4811-2764-4192-9e5c-db7fe3d7c06d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:32.536926 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca20e26-2719-485c-bb87-8468fb126988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:32.537571 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca20e26-2719-485c-bb87-8468fb126988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:32.537850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca20e26-2719-485c-bb87-8468fb126988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:32.538148 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca20e26-2719-485c-bb87-8468fb126988 admin admin] PATH_INFO: `/v3/projects/773ed98417064aa18010359755de111b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:32.539270 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca20e26-2719-485c-bb87-8468fb126988 admin admin] RBAC: Authorizing `identity:delete_project(project_id=773ed98417064aa18010359755de111b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:32.541204 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca20e26-2719-485c-bb87-8468fb126988 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:32.590911 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5ca20e26-2719-485c-bb87-8468fb126988 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '773ed98417064aa18010359755de111b', 'request_id': 'req-5ca20e26-2719-485c-bb87-8468fb126988'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:32.597773 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5ca20e26-2719-485c-bb87-8468fb126988 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '773ed98417064aa18010359755de111b', 'request_id': 'req-5ca20e26-2719-485c-bb87-8468fb126988'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:32.605088 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5ca20e26-2719-485c-bb87-8468fb126988 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '773ed98417064aa18010359755de111b', 'request_id': 'req-5ca20e26-2719-485c-bb87-8468fb126988'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:32.615658 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5ca20e26-2719-485c-bb87-8468fb126988 admin admin] The token cache is being invalidate because project 773ed98417064aa18010359755de111b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:43:32.615947 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5ca20e26-2719-485c-bb87-8468fb126988 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:32.616944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-5ca20e26-2719-485c-bb87-8468fb126988 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:32.618795 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1210/2426] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:43:32 2022] DELETE /identity/v3/projects/773ed98417064aa18010359755de111b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:39.488449 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7da96e-5064-4d09-8515-79103869e569 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:39.489888 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-ed7da96e-5064-4d09-8515-79103869e569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:39.503266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7da96e-5064-4d09-8515-79103869e569 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:39.565952 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75cfa3-afff-4c0f-a3ad-8d9bd0023c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:39.566588 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75cfa3-afff-4c0f-a3ad-8d9bd0023c05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:39.566872 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75cfa3-afff-4c0f-a3ad-8d9bd0023c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:39.567151 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75cfa3-afff-4c0f-a3ad-8d9bd0023c05 admin admin] PATH_INFO: `/v3/users/9062e47c8a22450db428554687f1f707` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:39.568278 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc75cfa3-afff-4c0f-a3ad-8d9bd0023c05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9062e47c8a22450db428554687f1f707)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:39.570247 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc75cfa3-afff-4c0f-a3ad-8d9bd0023c05 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:39.617570 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-fc75cfa3-afff-4c0f-a3ad-8d9bd0023c05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9062e47c8a22450db428554687f1f707'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:39.618225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-fc75cfa3-afff-4c0f-a3ad-8d9bd0023c05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9062e47c8a22450db428554687f1f707'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:39.625309 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-fc75cfa3-afff-4c0f-a3ad-8d9bd0023c05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9062e47c8a22450db428554687f1f707'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:39.625929 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-fc75cfa3-afff-4c0f-a3ad-8d9bd0023c05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9062e47c8a22450db428554687f1f707'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:39.633570 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-fc75cfa3-afff-4c0f-a3ad-8d9bd0023c05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9062e47c8a22450db428554687f1f707'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:39.651936 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1217/2427] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:43:39 2022] DELETE /identity/v3/users/9062e47c8a22450db428554687f1f707 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:39.656424 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfaaf89-0243-4d3c-9afe-a376d84f9b31 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:39.656957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfaaf89-0243-4d3c-9afe-a376d84f9b31 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:39.657083 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfaaf89-0243-4d3c-9afe-a376d84f9b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:39.686397 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-0cfaaf89-0243-4d3c-9afe-a376d84f9b31 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:39.687540 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0cfaaf89-0243-4d3c-9afe-a376d84f9b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:39.755447 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1211/2428] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:39.766015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75cfa3-afff-4c0f-a3ad-8d9bd0023c05 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:39.767633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-fc75cfa3-afff-4c0f-a3ad-8d9bd0023c05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:39.842244 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:39.846646 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75cfa3-afff-4c0f-a3ad-8d9bd0023c05 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:39.851703 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ba394-9a11-4e76-83b4-82d52a911257 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:39.852448 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ba394-9a11-4e76-83b4-82d52a911257 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:39.852772 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ba394-9a11-4e76-83b4-82d52a911257 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:39.853215 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ba394-9a11-4e76-83b4-82d52a911257 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:39.862816 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ba394-9a11-4e76-83b4-82d52a911257 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:39.864848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ba394-9a11-4e76-83b4-82d52a911257 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:39.874679 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1218/2429] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:43:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:43:40.144366 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfaaf89-0243-4d3c-9afe-a376d84f9b31 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:40.145932 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-0cfaaf89-0243-4d3c-9afe-a376d84f9b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:40.157624 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfaaf89-0243-4d3c-9afe-a376d84f9b31 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:40.162716 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec5aaf6-dc68-4c2f-a54f-90e4ea2ef99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:40.163346 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5aaf6-dc68-4c2f-a54f-90e4ea2ef99d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:40.163625 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5aaf6-dc68-4c2f-a54f-90e4ea2ef99d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:40.163890 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5aaf6-dc68-4c2f-a54f-90e4ea2ef99d admin admin] PATH_INFO: `/v3/projects/d15d68a115e44592aa896a797c2d0130` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:40.165058 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec5aaf6-dc68-4c2f-a54f-90e4ea2ef99d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d15d68a115e44592aa896a797c2d0130)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:40.166946 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec5aaf6-dc68-4c2f-a54f-90e4ea2ef99d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:40.175879 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:40.218650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fec5aaf6-dc68-4c2f-a54f-90e4ea2ef99d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd15d68a115e44592aa896a797c2d0130', 'request_id': 'req-fec5aaf6-dc68-4c2f-a54f-90e4ea2ef99d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:40.230324 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fec5aaf6-dc68-4c2f-a54f-90e4ea2ef99d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd15d68a115e44592aa896a797c2d0130', 'request_id': 'req-fec5aaf6-dc68-4c2f-a54f-90e4ea2ef99d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:40.238449 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fec5aaf6-dc68-4c2f-a54f-90e4ea2ef99d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd15d68a115e44592aa896a797c2d0130', 'request_id': 'req-fec5aaf6-dc68-4c2f-a54f-90e4ea2ef99d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:40.249186 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fec5aaf6-dc68-4c2f-a54f-90e4ea2ef99d admin admin] The token cache is being invalidate because project d15d68a115e44592aa896a797c2d0130 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:43:40.249474 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fec5aaf6-dc68-4c2f-a54f-90e4ea2ef99d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:40.250106 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-fec5aaf6-dc68-4c2f-a54f-90e4ea2ef99d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:40.252042 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1212/2430] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:43:40 2022] DELETE /identity/v3/projects/d15d68a115e44592aa896a797c2d0130 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:40.256500 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ba394-9a11-4e76-83b4-82d52a911257 service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:40.266684 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ba394-9a11-4e76-83b4-82d52a911257 service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:40.326450 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a014bf-0e70-4e43-b207-821678736a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:40.327081 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a014bf-0e70-4e43-b207-821678736a4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:40.327359 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a014bf-0e70-4e43-b207-821678736a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:40.327650 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a014bf-0e70-4e43-b207-821678736a4b admin admin] PATH_INFO: `/v3/users/c767f5f5732e4fafaf9a5576f3683d67` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:40.328739 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a014bf-0e70-4e43-b207-821678736a4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c767f5f5732e4fafaf9a5576f3683d67)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:40.331169 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a014bf-0e70-4e43-b207-821678736a4b admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:40.378620 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-79a014bf-0e70-4e43-b207-821678736a4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c767f5f5732e4fafaf9a5576f3683d67'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:40.379528 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-79a014bf-0e70-4e43-b207-821678736a4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c767f5f5732e4fafaf9a5576f3683d67'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:40.388227 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-79a014bf-0e70-4e43-b207-821678736a4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c767f5f5732e4fafaf9a5576f3683d67'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:40.389185 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-79a014bf-0e70-4e43-b207-821678736a4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c767f5f5732e4fafaf9a5576f3683d67'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:40.396995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-79a014bf-0e70-4e43-b207-821678736a4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c767f5f5732e4fafaf9a5576f3683d67'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:40.414551 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1219/2431] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:43:40 2022] DELETE /identity/v3/users/c767f5f5732e4fafaf9a5576f3683d67 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:40.694094 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec5aaf6-dc68-4c2f-a54f-90e4ea2ef99d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:40.695786 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-fec5aaf6-dc68-4c2f-a54f-90e4ea2ef99d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:40.711660 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec5aaf6-dc68-4c2f-a54f-90e4ea2ef99d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:40.774178 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b190a5-41ce-49b0-a3a9-99aa91931004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:40.774834 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b190a5-41ce-49b0-a3a9-99aa91931004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:40.775118 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b190a5-41ce-49b0-a3a9-99aa91931004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:40.775427 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b190a5-41ce-49b0-a3a9-99aa91931004 admin admin] PATH_INFO: `/v3/projects/d7e35c2ecf9244f39f909da7b43cc29d` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:40.776580 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b190a5-41ce-49b0-a3a9-99aa91931004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7e35c2ecf9244f39f909da7b43cc29d)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:40.778550 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b190a5-41ce-49b0-a3a9-99aa91931004 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:40.826611 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f1b190a5-41ce-49b0-a3a9-99aa91931004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7e35c2ecf9244f39f909da7b43cc29d', 'request_id': 'req-f1b190a5-41ce-49b0-a3a9-99aa91931004'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:40.833502 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f1b190a5-41ce-49b0-a3a9-99aa91931004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7e35c2ecf9244f39f909da7b43cc29d', 'request_id': 'req-f1b190a5-41ce-49b0-a3a9-99aa91931004'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:40.841183 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f1b190a5-41ce-49b0-a3a9-99aa91931004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7e35c2ecf9244f39f909da7b43cc29d', 'request_id': 'req-f1b190a5-41ce-49b0-a3a9-99aa91931004'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:40.851470 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f1b190a5-41ce-49b0-a3a9-99aa91931004 admin admin] The token cache is being invalidate because project d7e35c2ecf9244f39f909da7b43cc29d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:43:40.851755 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f1b190a5-41ce-49b0-a3a9-99aa91931004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:40.852372 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f1b190a5-41ce-49b0-a3a9-99aa91931004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:40.853914 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1213/2432] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:43:40 2022] DELETE /identity/v3/projects/d7e35c2ecf9244f39f909da7b43cc29d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:40.858704 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40e6d6-e01f-4ebc-9d97-72a7a65b4f66 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:40.859182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40e6d6-e01f-4ebc-9d97-72a7a65b4f66 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:40.859296 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40e6d6-e01f-4ebc-9d97-72a7a65b4f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:40.907659 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-4d40e6d6-e01f-4ebc-9d97-72a7a65b4f66 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:40.908843 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-4d40e6d6-e01f-4ebc-9d97-72a7a65b4f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:40.950820 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:40.971651 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1220/2433] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:40.980212 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa5942-a1c7-46db-a8cb-3fda32024a28 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:40.980518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa5942-a1c7-46db-a8cb-3fda32024a28 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:40.980791 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa5942-a1c7-46db-a8cb-3fda32024a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:41.000661 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-49aa5942-a1c7-46db-a8cb-3fda32024a28 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:41.002505 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-49aa5942-a1c7-46db-a8cb-3fda32024a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:41.010340 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1214/2434] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:41.014599 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40e6d6-e01f-4ebc-9d97-72a7a65b4f66 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:41.024648 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40e6d6-e01f-4ebc-9d97-72a7a65b4f66 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:41.029267 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e736f773-70ba-44ce-954d-e86c936384ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:41.029860 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736f773-70ba-44ce-954d-e86c936384ae admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:41.030144 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736f773-70ba-44ce-954d-e86c936384ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:41.030422 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736f773-70ba-44ce-954d-e86c936384ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:41.031070 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e736f773-70ba-44ce-954d-e86c936384ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81474) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:43:41.031377 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e736f773-70ba-44ce-954d-e86c936384ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:41.033004 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e736f773-70ba-44ce-954d-e86c936384ae admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:41.044456 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1221/2435] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:43:41.049219 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa5942-a1c7-46db-a8cb-3fda32024a28 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:41.058346 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa5942-a1c7-46db-a8cb-3fda32024a28 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:41.062956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9e255-ce47-435b-87f6-fe5cb59aa655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:41.063547 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9e255-ce47-435b-87f6-fe5cb59aa655 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:41.063827 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9e255-ce47-435b-87f6-fe5cb59aa655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:41.064636 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9e255-ce47-435b-87f6-fe5cb59aa655 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:41.065435 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b9e255-ce47-435b-87f6-fe5cb59aa655 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:41.067304 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b9e255-ce47-435b-87f6-fe5cb59aa655 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:41.087927 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:41.096234 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1215/2436] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:43:41 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:43:41.100244 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e736f773-70ba-44ce-954d-e86c936384ae admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:41.111234 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e736f773-70ba-44ce-954d-e86c936384ae admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:41.167166 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998361f-f6a2-4e08-ae55-5da66a2a485c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:41.167766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998361f-f6a2-4e08-ae55-5da66a2a485c admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:41.168044 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998361f-f6a2-4e08-ae55-5da66a2a485c admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:41.168321 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998361f-f6a2-4e08-ae55-5da66a2a485c admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:41.168981 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998361f-f6a2-4e08-ae55-5da66a2a485c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:41.170899 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998361f-f6a2-4e08-ae55-5da66a2a485c admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:41.192603 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1222/2437] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:43:41 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:43:41.197500 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9e255-ce47-435b-87f6-fe5cb59aa655 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:41.207987 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9e255-ce47-435b-87f6-fe5cb59aa655 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:41.212544 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5189a9-9bfe-4215-9480-636d8ab4b030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:41.213174 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5189a9-9bfe-4215-9480-636d8ab4b030 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:41.213455 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5189a9-9bfe-4215-9480-636d8ab4b030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:41.213726 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5189a9-9bfe-4215-9480-636d8ab4b030 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:41.214360 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5189a9-9bfe-4215-9480-636d8ab4b030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:41.215929 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5189a9-9bfe-4215-9480-636d8ab4b030 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:41.225744 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1216/2438] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:43:41.229711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998361f-f6a2-4e08-ae55-5da66a2a485c admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:41.240398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998361f-f6a2-4e08-ae55-5da66a2a485c admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:41.245230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf801d-8c6f-4060-9872-9a06ece14af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:41.245816 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf801d-8c6f-4060-9872-9a06ece14af6 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:41.246111 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf801d-8c6f-4060-9872-9a06ece14af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:41.246360 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf801d-8c6f-4060-9872-9a06ece14af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:41.246997 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf801d-8c6f-4060-9872-9a06ece14af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:41.248567 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf801d-8c6f-4060-9872-9a06ece14af6 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:41.259708 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1223/2439] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:43:41.263810 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5189a9-9bfe-4215-9480-636d8ab4b030 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:41.273776 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5189a9-9bfe-4215-9480-636d8ab4b030 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:41.278302 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d65cb1-3ce6-41fb-860a-a0c8ad85eb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:41.278952 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d65cb1-3ce6-41fb-860a-a0c8ad85eb96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:41.279237 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d65cb1-3ce6-41fb-860a-a0c8ad85eb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:41.279516 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d65cb1-3ce6-41fb-860a-a0c8ad85eb96 admin admin] PATH_INFO: `/v3/projects/9eed2388944f40ca81511a300cabab0d/users/56af4724be0e42acbb9320f37567cac8/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:41.294014 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d65cb1-3ce6-41fb-860a-a0c8ad85eb96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eed2388944f40ca81511a300cabab0d, user_id=56af4724be0e42acbb9320f37567cac8, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:41.296705 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d65cb1-3ce6-41fb-860a-a0c8ad85eb96 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:41.309806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1217/2440] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:43:41 2022] PUT /identity/v3/projects/9eed2388944f40ca81511a300cabab0d/users/56af4724be0e42acbb9320f37567cac8/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:41.375445 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa689af7-b4cd-4259-b33e-5a18c0ab8394 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:41.375734 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa689af7-b4cd-4259-b33e-5a18c0ab8394 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:41.376015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa689af7-b4cd-4259-b33e-5a18c0ab8394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:41.403928 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-aa689af7-b4cd-4259-b33e-5a18c0ab8394 None None] MFA Rules not processed for user `56af4724be0e42acbb9320f37567cac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:41.405164 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-aa689af7-b4cd-4259-b33e-5a18c0ab8394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:41.467446 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1224/2441] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:41.471922 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d65cb1-3ce6-41fb-860a-a0c8ad85eb96 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:41.536448 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d65cb1-3ce6-41fb-860a-a0c8ad85eb96 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:41.541038 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb14389-0baf-42a0-adda-85dcb0d84dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:41.541638 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb14389-0baf-42a0-adda-85dcb0d84dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:41.541917 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb14389-0baf-42a0-adda-85dcb0d84dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:41.542188 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb14389-0baf-42a0-adda-85dcb0d84dc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:41.542814 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb14389-0baf-42a0-adda-85dcb0d84dc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:41.544704 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb14389-0baf-42a0-adda-85dcb0d84dc0 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:41.575087 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1218/2442] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:43:41 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:43:41.576233 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa689af7-b4cd-4259-b33e-5a18c0ab8394 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:41.587457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa689af7-b4cd-4259-b33e-5a18c0ab8394 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:41.628483 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:41.645793 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28cda4-f14f-49df-852d-7ec67aa0b591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:41.646390 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28cda4-f14f-49df-852d-7ec67aa0b591 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:41.646667 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28cda4-f14f-49df-852d-7ec67aa0b591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:41.646937 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28cda4-f14f-49df-852d-7ec67aa0b591 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:41.647567 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28cda4-f14f-49df-852d-7ec67aa0b591 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:41.649529 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28cda4-f14f-49df-852d-7ec67aa0b591 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:41.671474 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1225/2443] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:43:41 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:43:41.674859 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb14389-0baf-42a0-adda-85dcb0d84dc0 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:41.685291 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb14389-0baf-42a0-adda-85dcb0d84dc0 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:41.690459 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83d123-85eb-4504-b914-7892d38fa099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:41.691054 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83d123-85eb-4504-b914-7892d38fa099 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:41.691338 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83d123-85eb-4504-b914-7892d38fa099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:41.691615 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83d123-85eb-4504-b914-7892d38fa099 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:41.692253 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83d123-85eb-4504-b914-7892d38fa099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:41.693879 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83d123-85eb-4504-b914-7892d38fa099 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:41.704404 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1219/2444] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:43:41.708617 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28cda4-f14f-49df-852d-7ec67aa0b591 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:41.718383 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28cda4-f14f-49df-852d-7ec67aa0b591 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:41.723694 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1e1b1-2ea0-4355-a7fb-61ecd065508a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:41.724405 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1e1b1-2ea0-4355-a7fb-61ecd065508a admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:41.724710 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1e1b1-2ea0-4355-a7fb-61ecd065508a admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:41.725020 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1e1b1-2ea0-4355-a7fb-61ecd065508a admin admin] PATH_INFO: `/v3/projects/38ef1b077fa4416a9fa41a6e0c62eb4e/users/94a354b7a1cb4b4abf6d8c5080d6c518/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:41.740429 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1e1b1-2ea0-4355-a7fb-61ecd065508a admin admin] RBAC: Authorizing `identity:create_grant(project_id=38ef1b077fa4416a9fa41a6e0c62eb4e, user_id=94a354b7a1cb4b4abf6d8c5080d6c518, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:41.743386 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1e1b1-2ea0-4355-a7fb-61ecd065508a admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:41.756215 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1226/2445] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:43:41 2022] PUT /identity/v3/projects/38ef1b077fa4416a9fa41a6e0c62eb4e/users/94a354b7a1cb4b4abf6d8c5080d6c518/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:41.760098 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83d123-85eb-4504-b914-7892d38fa099 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:41.770078 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83d123-85eb-4504-b914-7892d38fa099 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:41.831374 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4edeb2-4f2a-4c93-9142-65ed8f18736f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:41.831992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4edeb2-4f2a-4c93-9142-65ed8f18736f admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:41.832307 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4edeb2-4f2a-4c93-9142-65ed8f18736f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:41.832582 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4edeb2-4f2a-4c93-9142-65ed8f18736f admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:41.833265 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4edeb2-4f2a-4c93-9142-65ed8f18736f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:41.834845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4edeb2-4f2a-4c93-9142-65ed8f18736f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:41.844816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1220/2446] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:43:41.849459 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1e1b1-2ea0-4355-a7fb-61ecd065508a admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:41.859476 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1e1b1-2ea0-4355-a7fb-61ecd065508a admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:41.864923 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8949e6-153e-4b85-9cd1-794b489b2b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:41.865663 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8949e6-153e-4b85-9cd1-794b489b2b77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:41.866225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8949e6-153e-4b85-9cd1-794b489b2b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:41.866510 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8949e6-153e-4b85-9cd1-794b489b2b77 admin admin] PATH_INFO: `/v3/projects/38ef1b077fa4416a9fa41a6e0c62eb4e/users/94a354b7a1cb4b4abf6d8c5080d6c518/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:41.868389 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8949e6-153e-4b85-9cd1-794b489b2b77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38ef1b077fa4416a9fa41a6e0c62eb4e, user_id=94a354b7a1cb4b4abf6d8c5080d6c518, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:41.871090 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8949e6-153e-4b85-9cd1-794b489b2b77 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:41.881592 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1227/2447] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:43:41 2022] PUT /identity/v3/projects/38ef1b077fa4416a9fa41a6e0c62eb4e/users/94a354b7a1cb4b4abf6d8c5080d6c518/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:41.891242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf447740-eb5b-4c55-9454-4c6ce8ae7ce9 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:41.891520 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf447740-eb5b-4c55-9454-4c6ce8ae7ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:41.891801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf447740-eb5b-4c55-9454-4c6ce8ae7ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:41.919824 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-bf447740-eb5b-4c55-9454-4c6ce8ae7ce9 None None] MFA Rules not processed for user `94a354b7a1cb4b4abf6d8c5080d6c518`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:41.921066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-bf447740-eb5b-4c55-9454-4c6ce8ae7ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:41.941366 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:41.997401 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1221/2448] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:42.005541 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8949e6-153e-4b85-9cd1-794b489b2b77 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:42.007195 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-5e8949e6-153e-4b85-9cd1-794b489b2b77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:42.075102 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8949e6-153e-4b85-9cd1-794b489b2b77 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:42.078810 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f52b6-a43a-426f-aa08-d6549cce369a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:42.079408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f52b6-a43a-426f-aa08-d6549cce369a service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:42.080219 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f52b6-a43a-426f-aa08-d6549cce369a service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:42.080484 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f52b6-a43a-426f-aa08-d6549cce369a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:42.089328 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f52b6-a43a-426f-aa08-d6549cce369a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:42.090754 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f52b6-a43a-426f-aa08-d6549cce369a service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:42.099909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1228/2449] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:43:42 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:43:42.412798 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf447740-eb5b-4c55-9454-4c6ce8ae7ce9 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:42.421897 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf447740-eb5b-4c55-9454-4c6ce8ae7ce9 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:42.426462 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd7cb0-f0a9-4655-a3a0-a1b1ea2f883d tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94a354b7a1cb4b4abf6d8c5080d6c518', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38ef1b077fa4416a9fa41a6e0c62eb4e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:42.427066 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd7cb0-f0a9-4655-a3a0-a1b1ea2f883d tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:42.427348 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd7cb0-f0a9-4655-a3a0-a1b1ea2f883d tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:42.427623 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd7cb0-f0a9-4655-a3a0-a1b1ea2f883d tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:42.428267 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd7cb0-f0a9-4655-a3a0-a1b1ea2f883d tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:43:42.428570 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd7cb0-f0a9-4655-a3a0-a1b1ea2f883d tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:42.430188 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd7cb0-f0a9-4655-a3a0-a1b1ea2f883d tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:42.441918 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1222/2450] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:43:42.445857 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f52b6-a43a-426f-aa08-d6549cce369a service cinder] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:42.455386 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f52b6-a43a-426f-aa08-d6549cce369a service cinder] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:42.459845 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185452e-3495-4c1f-88a1-95b2996d94de tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94a354b7a1cb4b4abf6d8c5080d6c518', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38ef1b077fa4416a9fa41a6e0c62eb4e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:42.460423 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185452e-3495-4c1f-88a1-95b2996d94de tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:42.460701 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185452e-3495-4c1f-88a1-95b2996d94de tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:42.461004 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185452e-3495-4c1f-88a1-95b2996d94de tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:42.461622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185452e-3495-4c1f-88a1-95b2996d94de tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:42.463500 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185452e-3495-4c1f-88a1-95b2996d94de tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:42.489346 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1229/2451] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:43:42 2022] POST /identity/v3/projects => generated 353 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:43:42.493575 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd7cb0-f0a9-4655-a3a0-a1b1ea2f883d tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:42.504458 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd7cb0-f0a9-4655-a3a0-a1b1ea2f883d tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:42.559859 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e8a10-78fb-4a73-9a09-76eb3046abe8 tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94a354b7a1cb4b4abf6d8c5080d6c518', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38ef1b077fa4416a9fa41a6e0c62eb4e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:42.560458 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e8a10-78fb-4a73-9a09-76eb3046abe8 tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:42.560735 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e8a10-78fb-4a73-9a09-76eb3046abe8 tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:42.561044 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e8a10-78fb-4a73-9a09-76eb3046abe8 tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:42.561682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e8a10-78fb-4a73-9a09-76eb3046abe8 tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:43:42.561982 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e8a10-78fb-4a73-9a09-76eb3046abe8 tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:42.563545 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e8a10-78fb-4a73-9a09-76eb3046abe8 tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:42.575333 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1223/2452] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:43:42.612643 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185452e-3495-4c1f-88a1-95b2996d94de tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:42.622826 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185452e-3495-4c1f-88a1-95b2996d94de tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:42.628070 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082d94d-40e4-4aba-9c12-9a6e09e48364 tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94a354b7a1cb4b4abf6d8c5080d6c518', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38ef1b077fa4416a9fa41a6e0c62eb4e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:42.628700 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082d94d-40e4-4aba-9c12-9a6e09e48364 tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:42.629015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082d94d-40e4-4aba-9c12-9a6e09e48364 tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:42.629288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082d94d-40e4-4aba-9c12-9a6e09e48364 tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] PATH_INFO: `/v3/projects/a75b2a86441840abab829177468da042` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:42.630398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082d94d-40e4-4aba-9c12-9a6e09e48364 tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a75b2a86441840abab829177468da042)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:42.632288 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082d94d-40e4-4aba-9c12-9a6e09e48364 tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:42.681097 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4082d94d-40e4-4aba-9c12-9a6e09e48364 tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a75b2a86441840abab829177468da042', 'request_id': 'req-4082d94d-40e4-4aba-9c12-9a6e09e48364'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:42.687684 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4082d94d-40e4-4aba-9c12-9a6e09e48364 tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a75b2a86441840abab829177468da042', 'request_id': 'req-4082d94d-40e4-4aba-9c12-9a6e09e48364'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:42.695547 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4082d94d-40e4-4aba-9c12-9a6e09e48364 tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a75b2a86441840abab829177468da042', 'request_id': 'req-4082d94d-40e4-4aba-9c12-9a6e09e48364'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:42.702219 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:42.707575 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4082d94d-40e4-4aba-9c12-9a6e09e48364 tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] The token cache is being invalidate because project a75b2a86441840abab829177468da042 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:43:42.708304 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4082d94d-40e4-4aba-9c12-9a6e09e48364 tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:42.709267 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4082d94d-40e4-4aba-9c12-9a6e09e48364 tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:42.710993 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1230/2453] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:43:42 2022] DELETE /identity/v3/projects/a75b2a86441840abab829177468da042 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:42.910008 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e8a10-78fb-4a73-9a09-76eb3046abe8 tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:42.911923 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2e3e8a10-78fb-4a73-9a09-76eb3046abe8 tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:42.978937 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e8a10-78fb-4a73-9a09-76eb3046abe8 tempest-VolumeQuotaClassesTest-1956099725 tempest-VolumeQuotaClassesTest-1956099725-project-admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:43.028657 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:43.044556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee971e9a-f5b5-4c55-a60b-17bed60b705b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:43.045875 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee971e9a-f5b5-4c55-a60b-17bed60b705b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:43.046165 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee971e9a-f5b5-4c55-a60b-17bed60b705b admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:43.046438 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee971e9a-f5b5-4c55-a60b-17bed60b705b admin admin] PATH_INFO: `/v3/users/56af4724be0e42acbb9320f37567cac8` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:43.047609 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee971e9a-f5b5-4c55-a60b-17bed60b705b admin admin] RBAC: Authorizing `identity:delete_user(user_id=56af4724be0e42acbb9320f37567cac8)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:43.050342 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee971e9a-f5b5-4c55-a60b-17bed60b705b admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:43.112672 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ee971e9a-f5b5-4c55-a60b-17bed60b705b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56af4724be0e42acbb9320f37567cac8'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:43.113600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ee971e9a-f5b5-4c55-a60b-17bed60b705b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56af4724be0e42acbb9320f37567cac8'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:43.121132 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ee971e9a-f5b5-4c55-a60b-17bed60b705b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56af4724be0e42acbb9320f37567cac8'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:43.129747 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ee971e9a-f5b5-4c55-a60b-17bed60b705b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56af4724be0e42acbb9320f37567cac8'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:43.130461 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-ee971e9a-f5b5-4c55-a60b-17bed60b705b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56af4724be0e42acbb9320f37567cac8'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:43.149408 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1224/2454] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:43:42 2022] DELETE /identity/v3/users/56af4724be0e42acbb9320f37567cac8 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:43.153663 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf309f1-96f2-4221-837a-a79e7c7ba7ba None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:43.154159 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf309f1-96f2-4221-837a-a79e7c7ba7ba None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:43.154271 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf309f1-96f2-4221-837a-a79e7c7ba7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:43.174191 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-edf309f1-96f2-4221-837a-a79e7c7ba7ba None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:43.175400 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-edf309f1-96f2-4221-837a-a79e7c7ba7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:43.241845 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1231/2455] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:43.253002 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee971e9a-f5b5-4c55-a60b-17bed60b705b admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:43.254552 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ee971e9a-f5b5-4c55-a60b-17bed60b705b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:43.316457 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee971e9a-f5b5-4c55-a60b-17bed60b705b admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:43.319471 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1781b9-cb59-4c26-b197-9b24fedc19df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:43.320180 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1781b9-cb59-4c26-b197-9b24fedc19df service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:43.320522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1781b9-cb59-4c26-b197-9b24fedc19df service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:43.320853 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1781b9-cb59-4c26-b197-9b24fedc19df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:43.330405 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1781b9-cb59-4c26-b197-9b24fedc19df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:43.331827 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1781b9-cb59-4c26-b197-9b24fedc19df service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:43.342400 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1225/2456] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:43:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:43:43.639917 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf309f1-96f2-4221-837a-a79e7c7ba7ba None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:43.646277 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-edf309f1-96f2-4221-837a-a79e7c7ba7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:43.676430 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf309f1-96f2-4221-837a-a79e7c7ba7ba None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:43.683747 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0573c-3787-4209-892a-e5b0a94e586a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:43.684821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0573c-3787-4209-892a-e5b0a94e586a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:43.684821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0573c-3787-4209-892a-e5b0a94e586a admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:43.685264 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0573c-3787-4209-892a-e5b0a94e586a admin admin] PATH_INFO: `/v3/projects/9eed2388944f40ca81511a300cabab0d` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:43.686482 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0573c-3787-4209-892a-e5b0a94e586a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eed2388944f40ca81511a300cabab0d)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:43.688490 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0573c-3787-4209-892a-e5b0a94e586a admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:43.744449 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5fc0573c-3787-4209-892a-e5b0a94e586a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eed2388944f40ca81511a300cabab0d', 'request_id': 'req-5fc0573c-3787-4209-892a-e5b0a94e586a'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:43.750872 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5fc0573c-3787-4209-892a-e5b0a94e586a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eed2388944f40ca81511a300cabab0d', 'request_id': 'req-5fc0573c-3787-4209-892a-e5b0a94e586a'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:43.759553 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5fc0573c-3787-4209-892a-e5b0a94e586a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eed2388944f40ca81511a300cabab0d', 'request_id': 'req-5fc0573c-3787-4209-892a-e5b0a94e586a'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:43.770996 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5fc0573c-3787-4209-892a-e5b0a94e586a admin admin] The token cache is being invalidate because project 9eed2388944f40ca81511a300cabab0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:43:43.771281 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5fc0573c-3787-4209-892a-e5b0a94e586a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:43.771895 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-5fc0573c-3787-4209-892a-e5b0a94e586a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:43.773414 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1232/2457] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:43:43 2022] DELETE /identity/v3/projects/9eed2388944f40ca81511a300cabab0d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:43.777562 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1781b9-cb59-4c26-b197-9b24fedc19df service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:43.787879 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1781b9-cb59-4c26-b197-9b24fedc19df service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:43.842800 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82aa056-7875-490d-b43b-3ca627cc74f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:43.843420 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82aa056-7875-490d-b43b-3ca627cc74f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:43.843699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82aa056-7875-490d-b43b-3ca627cc74f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:43.843971 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82aa056-7875-490d-b43b-3ca627cc74f2 admin admin] PATH_INFO: `/v3/users/94a354b7a1cb4b4abf6d8c5080d6c518` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:43.845885 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82aa056-7875-490d-b43b-3ca627cc74f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94a354b7a1cb4b4abf6d8c5080d6c518)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:43.847936 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82aa056-7875-490d-b43b-3ca627cc74f2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:43.857503 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:43.896913 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a82aa056-7875-490d-b43b-3ca627cc74f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94a354b7a1cb4b4abf6d8c5080d6c518'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:43.897440 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a82aa056-7875-490d-b43b-3ca627cc74f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94a354b7a1cb4b4abf6d8c5080d6c518'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:43.905539 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a82aa056-7875-490d-b43b-3ca627cc74f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94a354b7a1cb4b4abf6d8c5080d6c518'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:43.914842 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a82aa056-7875-490d-b43b-3ca627cc74f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94a354b7a1cb4b4abf6d8c5080d6c518'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:43.915742 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-a82aa056-7875-490d-b43b-3ca627cc74f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94a354b7a1cb4b4abf6d8c5080d6c518'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:43.934471 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1226/2458] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:43:43 2022] DELETE /identity/v3/users/94a354b7a1cb4b4abf6d8c5080d6c518 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:44.215934 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0573c-3787-4209-892a-e5b0a94e586a admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:44.217477 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-5fc0573c-3787-4209-892a-e5b0a94e586a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:44.230349 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0573c-3787-4209-892a-e5b0a94e586a admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:44.288680 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77fedbc-1ab1-4d40-8162-01cc13c3b887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:44.289342 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fedbc-1ab1-4d40-8162-01cc13c3b887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:44.289622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fedbc-1ab1-4d40-8162-01cc13c3b887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:44.289896 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77fedbc-1ab1-4d40-8162-01cc13c3b887 admin admin] PATH_INFO: `/v3/projects/38ef1b077fa4416a9fa41a6e0c62eb4e` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:44.291002 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77fedbc-1ab1-4d40-8162-01cc13c3b887 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38ef1b077fa4416a9fa41a6e0c62eb4e)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:44.292930 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77fedbc-1ab1-4d40-8162-01cc13c3b887 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:44.317347 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:44.340646 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-d77fedbc-1ab1-4d40-8162-01cc13c3b887 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38ef1b077fa4416a9fa41a6e0c62eb4e', 'request_id': 'req-d77fedbc-1ab1-4d40-8162-01cc13c3b887'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:44.346857 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-d77fedbc-1ab1-4d40-8162-01cc13c3b887 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38ef1b077fa4416a9fa41a6e0c62eb4e', 'request_id': 'req-d77fedbc-1ab1-4d40-8162-01cc13c3b887'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:44.354471 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-d77fedbc-1ab1-4d40-8162-01cc13c3b887 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38ef1b077fa4416a9fa41a6e0c62eb4e', 'request_id': 'req-d77fedbc-1ab1-4d40-8162-01cc13c3b887'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:44.365770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-d77fedbc-1ab1-4d40-8162-01cc13c3b887 admin admin] The token cache is being invalidate because project 38ef1b077fa4416a9fa41a6e0c62eb4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:43:44.366338 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-d77fedbc-1ab1-4d40-8162-01cc13c3b887 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:44.366955 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-d77fedbc-1ab1-4d40-8162-01cc13c3b887 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:44.368398 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1233/2459] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:43:44 2022] DELETE /identity/v3/projects/38ef1b077fa4416a9fa41a6e0c62eb4e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:44.373245 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d834be-08b8-4507-af33-259a91664cb7 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:44.373532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d834be-08b8-4507-af33-259a91664cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:44.373807 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d834be-08b8-4507-af33-259a91664cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:44.393694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-a8d834be-08b8-4507-af33-259a91664cb7 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:44.394876 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a8d834be-08b8-4507-af33-259a91664cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:44.453578 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1227/2460] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:44.461292 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe177e-167d-4d03-bc90-7d420efb2994 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:44.461518 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe177e-167d-4d03-bc90-7d420efb2994 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:44.461789 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe177e-167d-4d03-bc90-7d420efb2994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:44.481389 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-b4fe177e-167d-4d03-bc90-7d420efb2994 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:44.482603 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-b4fe177e-167d-4d03-bc90-7d420efb2994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:44.491587 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1234/2461] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:44.495745 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d834be-08b8-4507-af33-259a91664cb7 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:44.506032 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d834be-08b8-4507-af33-259a91664cb7 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:44.510712 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864cf511-8904-4468-bb87-46ddf09d6667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:44.511343 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864cf511-8904-4468-bb87-46ddf09d6667 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:44.511568 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864cf511-8904-4468-bb87-46ddf09d6667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:44.511840 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864cf511-8904-4468-bb87-46ddf09d6667 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:44.512483 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864cf511-8904-4468-bb87-46ddf09d6667 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:43:44.512787 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864cf511-8904-4468-bb87-46ddf09d6667 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:44.514402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864cf511-8904-4468-bb87-46ddf09d6667 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:44.525902 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1228/2462] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:43:44.530221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe177e-167d-4d03-bc90-7d420efb2994 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:44.540361 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe177e-167d-4d03-bc90-7d420efb2994 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:44.544977 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e182132-5c3e-4e37-821c-4dda8d34e2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:44.545565 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e182132-5c3e-4e37-821c-4dda8d34e2b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:44.545843 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e182132-5c3e-4e37-821c-4dda8d34e2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:44.546121 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e182132-5c3e-4e37-821c-4dda8d34e2b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:44.546742 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e182132-5c3e-4e37-821c-4dda8d34e2b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:44.549238 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e182132-5c3e-4e37-821c-4dda8d34e2b4 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:44.576082 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1235/2463] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:43:44 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:43:44.580839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864cf511-8904-4468-bb87-46ddf09d6667 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:44.590642 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864cf511-8904-4468-bb87-46ddf09d6667 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:44.647074 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f825b9-b07d-4bc9-8024-1ca6fd4c004a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:44.647734 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f825b9-b07d-4bc9-8024-1ca6fd4c004a admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:44.648021 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f825b9-b07d-4bc9-8024-1ca6fd4c004a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:44.648295 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f825b9-b07d-4bc9-8024-1ca6fd4c004a admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:44.648970 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f825b9-b07d-4bc9-8024-1ca6fd4c004a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:44.650928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f825b9-b07d-4bc9-8024-1ca6fd4c004a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:44.671504 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1229/2464] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:43:44 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:43:44.676183 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e182132-5c3e-4e37-821c-4dda8d34e2b4 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:44.686287 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e182132-5c3e-4e37-821c-4dda8d34e2b4 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:44.692020 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14199007-f129-4be4-ac2d-4760d3a17529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:44.692619 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14199007-f129-4be4-ac2d-4760d3a17529 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:44.692944 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14199007-f129-4be4-ac2d-4760d3a17529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:44.693221 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14199007-f129-4be4-ac2d-4760d3a17529 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:44.693870 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14199007-f129-4be4-ac2d-4760d3a17529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:44.695461 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14199007-f129-4be4-ac2d-4760d3a17529 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:44.706128 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1236/2465] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:43:44.709955 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f825b9-b07d-4bc9-8024-1ca6fd4c004a admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:44.716302 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:44.720629 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f825b9-b07d-4bc9-8024-1ca6fd4c004a admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:44.726724 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c018769-8ba3-41fb-922f-16f9f68314f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:44.727356 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c018769-8ba3-41fb-922f-16f9f68314f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:44.727825 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c018769-8ba3-41fb-922f-16f9f68314f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:44.728287 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c018769-8ba3-41fb-922f-16f9f68314f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:44.728958 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c018769-8ba3-41fb-922f-16f9f68314f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:44.730543 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c018769-8ba3-41fb-922f-16f9f68314f3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:44.741627 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1230/2466] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:43:44.745656 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14199007-f129-4be4-ac2d-4760d3a17529 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:44.756603 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14199007-f129-4be4-ac2d-4760d3a17529 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:44.761190 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934daa81-2056-413a-aa44-d3cabcbeb5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:44.761833 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934daa81-2056-413a-aa44-d3cabcbeb5d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:44.762113 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934daa81-2056-413a-aa44-d3cabcbeb5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:44.762387 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934daa81-2056-413a-aa44-d3cabcbeb5d8 admin admin] PATH_INFO: `/v3/projects/77169c53a2ac423ba77a7631f6ce554c/users/ca9a55343c804b03843b85cac0f95a43/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:44.777999 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934daa81-2056-413a-aa44-d3cabcbeb5d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77169c53a2ac423ba77a7631f6ce554c, user_id=ca9a55343c804b03843b85cac0f95a43, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:44.780687 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934daa81-2056-413a-aa44-d3cabcbeb5d8 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:44.793735 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1237/2467] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:43:44 2022] PUT /identity/v3/projects/77169c53a2ac423ba77a7631f6ce554c/users/ca9a55343c804b03843b85cac0f95a43/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:44.805697 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75545e2b-1e10-4d46-abc0-d4d0948bd6b4 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:44.805981 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75545e2b-1e10-4d46-abc0-d4d0948bd6b4 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:44.806251 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75545e2b-1e10-4d46-abc0-d4d0948bd6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:44.834118 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-75545e2b-1e10-4d46-abc0-d4d0948bd6b4 None None] MFA Rules not processed for user `ca9a55343c804b03843b85cac0f95a43`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:44.835306 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-75545e2b-1e10-4d46-abc0-d4d0948bd6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:44.899879 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1231/2468] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:44.904310 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934daa81-2056-413a-aa44-d3cabcbeb5d8 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:44.966908 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934daa81-2056-413a-aa44-d3cabcbeb5d8 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:44.971693 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85046113-e27e-45e3-a1c5-cd14df54d55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:44.972301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85046113-e27e-45e3-a1c5-cd14df54d55d admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:44.972583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85046113-e27e-45e3-a1c5-cd14df54d55d admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:44.972856 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85046113-e27e-45e3-a1c5-cd14df54d55d admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:44.973528 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85046113-e27e-45e3-a1c5-cd14df54d55d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:44.975499 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85046113-e27e-45e3-a1c5-cd14df54d55d admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:45.011182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1238/2469] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:43:44 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:43:45.012346 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75545e2b-1e10-4d46-abc0-d4d0948bd6b4 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:45.023693 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75545e2b-1e10-4d46-abc0-d4d0948bd6b4 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:45.081632 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793693f-a77f-4084-9ec8-4afe1b944b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:45.082279 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793693f-a77f-4084-9ec8-4afe1b944b47 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:45.082558 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793693f-a77f-4084-9ec8-4afe1b944b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:45.082836 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793693f-a77f-4084-9ec8-4afe1b944b47 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:45.083467 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793693f-a77f-4084-9ec8-4afe1b944b47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:45.085387 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793693f-a77f-4084-9ec8-4afe1b944b47 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:45.107028 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1232/2470] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:43:45 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:43:45.111753 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85046113-e27e-45e3-a1c5-cd14df54d55d admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:45.122980 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85046113-e27e-45e3-a1c5-cd14df54d55d admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:45.128378 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87fe9f-fb97-43d3-b20b-86d93b4f6697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:45.129098 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87fe9f-fb97-43d3-b20b-86d93b4f6697 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:45.129443 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87fe9f-fb97-43d3-b20b-86d93b4f6697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:45.129780 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87fe9f-fb97-43d3-b20b-86d93b4f6697 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:45.130485 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87fe9f-fb97-43d3-b20b-86d93b4f6697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:45.132134 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87fe9f-fb97-43d3-b20b-86d93b4f6697 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:45.146882 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1239/2471] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:43:45.148136 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793693f-a77f-4084-9ec8-4afe1b944b47 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:45.158677 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793693f-a77f-4084-9ec8-4afe1b944b47 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:45.164151 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372b812-7403-4d71-9044-7221f9dafcc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:45.164837 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372b812-7403-4d71-9044-7221f9dafcc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:45.165697 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372b812-7403-4d71-9044-7221f9dafcc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:45.165697 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372b812-7403-4d71-9044-7221f9dafcc1 admin admin] PATH_INFO: `/v3/projects/d09e60e08a494a05940297c92740a601/users/1f7a6790de694d9a83a1f51787fe5249/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:45.181350 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372b812-7403-4d71-9044-7221f9dafcc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d09e60e08a494a05940297c92740a601, user_id=1f7a6790de694d9a83a1f51787fe5249, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:45.184034 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372b812-7403-4d71-9044-7221f9dafcc1 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:45.201839 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1233/2472] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:43:45 2022] PUT /identity/v3/projects/d09e60e08a494a05940297c92740a601/users/1f7a6790de694d9a83a1f51787fe5249/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:45.206201 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87fe9f-fb97-43d3-b20b-86d93b4f6697 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:45.216516 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87fe9f-fb97-43d3-b20b-86d93b4f6697 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:45.238703 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:45.283292 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abcebe3-6f11-4ec9-b339-8f7763e339af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:45.284044 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcebe3-6f11-4ec9-b339-8f7763e339af admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:45.284286 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcebe3-6f11-4ec9-b339-8f7763e339af admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:45.284562 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcebe3-6f11-4ec9-b339-8f7763e339af admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:45.285635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abcebe3-6f11-4ec9-b339-8f7763e339af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:45.299379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abcebe3-6f11-4ec9-b339-8f7763e339af admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:45.310000 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1240/2473] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:43:45.314978 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372b812-7403-4d71-9044-7221f9dafcc1 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:45.325584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372b812-7403-4d71-9044-7221f9dafcc1 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:45.331894 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f62f1-260e-4d50-b0c3-ffd934242855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:45.332652 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f62f1-260e-4d50-b0c3-ffd934242855 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:45.332981 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f62f1-260e-4d50-b0c3-ffd934242855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:45.333246 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f62f1-260e-4d50-b0c3-ffd934242855 admin admin] PATH_INFO: `/v3/projects/d09e60e08a494a05940297c92740a601/users/1f7a6790de694d9a83a1f51787fe5249/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:45.335522 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f62f1-260e-4d50-b0c3-ffd934242855 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d09e60e08a494a05940297c92740a601, user_id=1f7a6790de694d9a83a1f51787fe5249, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:45.338700 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f62f1-260e-4d50-b0c3-ffd934242855 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:45.351621 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1234/2474] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:43:45 2022] PUT /identity/v3/projects/d09e60e08a494a05940297c92740a601/users/1f7a6790de694d9a83a1f51787fe5249/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:43:45.361605 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cdfc5-627a-4a1e-8e19-203379119505 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:45.361892 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cdfc5-627a-4a1e-8e19-203379119505 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:45.362165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cdfc5-627a-4a1e-8e19-203379119505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:45.391356 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-8d9cdfc5-627a-4a1e-8e19-203379119505 None None] MFA Rules not processed for user `1f7a6790de694d9a83a1f51787fe5249`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:45.392770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-8d9cdfc5-627a-4a1e-8e19-203379119505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:45.460795 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1241/2475] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:45.472650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f62f1-260e-4d50-b0c3-ffd934242855 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:45.474361 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8c8f62f1-260e-4d50-b0c3-ffd934242855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:45.514821 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:45.533753 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f62f1-260e-4d50-b0c3-ffd934242855 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:45.537368 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d579d0-521d-4b90-be7f-b0854b3cad17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:45.537965 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d579d0-521d-4b90-be7f-b0854b3cad17 service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:45.538261 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d579d0-521d-4b90-be7f-b0854b3cad17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:45.538528 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d579d0-521d-4b90-be7f-b0854b3cad17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:45.547105 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d579d0-521d-4b90-be7f-b0854b3cad17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:45.548754 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d579d0-521d-4b90-be7f-b0854b3cad17 service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:45.557893 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1235/2476] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:43:45 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:43:45.644013 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cdfc5-627a-4a1e-8e19-203379119505 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:45.653758 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cdfc5-627a-4a1e-8e19-203379119505 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:45.657820 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430e114-fd01-44cb-a268-ada10b284f12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:45.658345 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430e114-fd01-44cb-a268-ada10b284f12 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:45.658627 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430e114-fd01-44cb-a268-ada10b284f12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:45.658902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430e114-fd01-44cb-a268-ada10b284f12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:45.717166 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430e114-fd01-44cb-a268-ada10b284f12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:45.718612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430e114-fd01-44cb-a268-ada10b284f12 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:45.727731 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1242/2477] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:43:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:43:51.477257 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d579d0-521d-4b90-be7f-b0854b3cad17 service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:51.550053 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d579d0-521d-4b90-be7f-b0854b3cad17 service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:51.554638 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3875d0f-c553-468e-bd37-8dfcf9cfb2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:51.555267 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3875d0f-c553-468e-bd37-8dfcf9cfb2f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:51.555547 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3875d0f-c553-468e-bd37-8dfcf9cfb2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:51.555821 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3875d0f-c553-468e-bd37-8dfcf9cfb2f7 admin admin] PATH_INFO: `/v3/users/ca9a55343c804b03843b85cac0f95a43` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:51.556983 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3875d0f-c553-468e-bd37-8dfcf9cfb2f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca9a55343c804b03843b85cac0f95a43)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:51.558880 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3875d0f-c553-468e-bd37-8dfcf9cfb2f7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:51.611241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b3875d0f-c553-468e-bd37-8dfcf9cfb2f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca9a55343c804b03843b85cac0f95a43'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:51.611241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b3875d0f-c553-468e-bd37-8dfcf9cfb2f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca9a55343c804b03843b85cac0f95a43'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:51.618682 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b3875d0f-c553-468e-bd37-8dfcf9cfb2f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca9a55343c804b03843b85cac0f95a43'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:51.626558 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b3875d0f-c553-468e-bd37-8dfcf9cfb2f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca9a55343c804b03843b85cac0f95a43'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:51.627360 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-b3875d0f-c553-468e-bd37-8dfcf9cfb2f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca9a55343c804b03843b85cac0f95a43'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:51.645497 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1236/2478] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:43:51 2022] DELETE /identity/v3/users/ca9a55343c804b03843b85cac0f95a43 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:51.650129 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7487601-da56-47f7-93b6-7df1ffd3b3df None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:51.650611 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7487601-da56-47f7-93b6-7df1ffd3b3df None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:51.656114 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7487601-da56-47f7-93b6-7df1ffd3b3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:51.670575 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-f7487601-da56-47f7-93b6-7df1ffd3b3df None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:51.671854 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-f7487601-da56-47f7-93b6-7df1ffd3b3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:51.711801 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:51.740283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1243/2479] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:51.747220 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3875d0f-c553-468e-bd37-8dfcf9cfb2f7 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:51.748697 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b3875d0f-c553-468e-bd37-8dfcf9cfb2f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:51.817859 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:51.837188 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3875d0f-c553-468e-bd37-8dfcf9cfb2f7 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:51.841134 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9fae6-fe53-40f2-82fa-c38e16ab5221 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:51.841812 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9fae6-fe53-40f2-82fa-c38e16ab5221 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:51.842166 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9fae6-fe53-40f2-82fa-c38e16ab5221 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:51.842351 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9fae6-fe53-40f2-82fa-c38e16ab5221 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:51.854024 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9fae6-fe53-40f2-82fa-c38e16ab5221 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:51.855534 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9fae6-fe53-40f2-82fa-c38e16ab5221 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:51.866240 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1237/2480] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:43:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:43:52.139302 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7487601-da56-47f7-93b6-7df1ffd3b3df None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:52.141182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-f7487601-da56-47f7-93b6-7df1ffd3b3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:52.159862 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7487601-da56-47f7-93b6-7df1ffd3b3df None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:52.165661 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5804d6-d414-41c4-acb4-8acd49cbf017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:52.166520 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5804d6-d414-41c4-acb4-8acd49cbf017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:52.166887 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5804d6-d414-41c4-acb4-8acd49cbf017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:52.167176 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5804d6-d414-41c4-acb4-8acd49cbf017 admin admin] PATH_INFO: `/v3/projects/77169c53a2ac423ba77a7631f6ce554c` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:52.168555 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5804d6-d414-41c4-acb4-8acd49cbf017 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77169c53a2ac423ba77a7631f6ce554c)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:52.170765 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5804d6-d414-41c4-acb4-8acd49cbf017 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:52.225029 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6f5804d6-d414-41c4-acb4-8acd49cbf017 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77169c53a2ac423ba77a7631f6ce554c', 'request_id': 'req-6f5804d6-d414-41c4-acb4-8acd49cbf017'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:52.231637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6f5804d6-d414-41c4-acb4-8acd49cbf017 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77169c53a2ac423ba77a7631f6ce554c', 'request_id': 'req-6f5804d6-d414-41c4-acb4-8acd49cbf017'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:52.241069 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6f5804d6-d414-41c4-acb4-8acd49cbf017 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77169c53a2ac423ba77a7631f6ce554c', 'request_id': 'req-6f5804d6-d414-41c4-acb4-8acd49cbf017'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:52.251793 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6f5804d6-d414-41c4-acb4-8acd49cbf017 admin admin] The token cache is being invalidate because project 77169c53a2ac423ba77a7631f6ce554c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:43:52.252086 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6f5804d6-d414-41c4-acb4-8acd49cbf017 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:52.252723 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6f5804d6-d414-41c4-acb4-8acd49cbf017 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:52.254300 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1244/2481] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:43:52 2022] DELETE /identity/v3/projects/77169c53a2ac423ba77a7631f6ce554c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:52.258935 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9fae6-fe53-40f2-82fa-c38e16ab5221 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:52.268329 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9fae6-fe53-40f2-82fa-c38e16ab5221 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:52.324549 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25e1d5-5370-493c-b085-025eb02758a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:52.325218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25e1d5-5370-493c-b085-025eb02758a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:52.325500 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25e1d5-5370-493c-b085-025eb02758a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:52.325780 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25e1d5-5370-493c-b085-025eb02758a4 admin admin] PATH_INFO: `/v3/users/1f7a6790de694d9a83a1f51787fe5249` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:52.326948 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25e1d5-5370-493c-b085-025eb02758a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f7a6790de694d9a83a1f51787fe5249)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:52.328926 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25e1d5-5370-493c-b085-025eb02758a4 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:52.377614 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-bb25e1d5-5370-493c-b085-025eb02758a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f7a6790de694d9a83a1f51787fe5249'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:52.378269 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-bb25e1d5-5370-493c-b085-025eb02758a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f7a6790de694d9a83a1f51787fe5249'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:52.385141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-bb25e1d5-5370-493c-b085-025eb02758a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f7a6790de694d9a83a1f51787fe5249'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:52.393435 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-bb25e1d5-5370-493c-b085-025eb02758a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f7a6790de694d9a83a1f51787fe5249'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:52.394181 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-bb25e1d5-5370-493c-b085-025eb02758a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f7a6790de694d9a83a1f51787fe5249'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:52.414121 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1238/2482] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:43:52 2022] DELETE /identity/v3/users/1f7a6790de694d9a83a1f51787fe5249 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:52.677557 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5804d6-d414-41c4-acb4-8acd49cbf017 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:52.679184 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-6f5804d6-d414-41c4-acb4-8acd49cbf017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:52.695290 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5804d6-d414-41c4-acb4-8acd49cbf017 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:52.754388 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce08b9a-5a65-4e38-b5a2-fdfeb1c68e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:52.755010 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce08b9a-5a65-4e38-b5a2-fdfeb1c68e86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:52.755290 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce08b9a-5a65-4e38-b5a2-fdfeb1c68e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:52.755559 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce08b9a-5a65-4e38-b5a2-fdfeb1c68e86 admin admin] PATH_INFO: `/v3/projects/d09e60e08a494a05940297c92740a601` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:52.756655 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce08b9a-5a65-4e38-b5a2-fdfeb1c68e86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d09e60e08a494a05940297c92740a601)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:52.758583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce08b9a-5a65-4e38-b5a2-fdfeb1c68e86 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:52.811896 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4ce08b9a-5a65-4e38-b5a2-fdfeb1c68e86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd09e60e08a494a05940297c92740a601', 'request_id': 'req-4ce08b9a-5a65-4e38-b5a2-fdfeb1c68e86'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:52.818751 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4ce08b9a-5a65-4e38-b5a2-fdfeb1c68e86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd09e60e08a494a05940297c92740a601', 'request_id': 'req-4ce08b9a-5a65-4e38-b5a2-fdfeb1c68e86'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:52.825852 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4ce08b9a-5a65-4e38-b5a2-fdfeb1c68e86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd09e60e08a494a05940297c92740a601', 'request_id': 'req-4ce08b9a-5a65-4e38-b5a2-fdfeb1c68e86'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:52.830821 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:52.838570 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4ce08b9a-5a65-4e38-b5a2-fdfeb1c68e86 admin admin] The token cache is being invalidate because project d09e60e08a494a05940297c92740a601 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:43:52.839051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4ce08b9a-5a65-4e38-b5a2-fdfeb1c68e86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:52.839781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4ce08b9a-5a65-4e38-b5a2-fdfeb1c68e86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:52.841502 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1245/2483] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:43:52 2022] DELETE /identity/v3/projects/d09e60e08a494a05940297c92740a601 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:52.847133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c23e3-aabc-4129-a876-95f85c466958 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:52.847617 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c23e3-aabc-4129-a876-95f85c466958 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:52.847716 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c23e3-aabc-4129-a876-95f85c466958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:52.867724 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-954c23e3-aabc-4129-a876-95f85c466958 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:52.868835 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-954c23e3-aabc-4129-a876-95f85c466958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:52.916986 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce08b9a-5a65-4e38-b5a2-fdfeb1c68e86 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:52.918590 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-4ce08b9a-5a65-4e38-b5a2-fdfeb1c68e86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:52.919104 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:52.930364 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1239/2484] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:43:52.939148 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c35a55-8d28-4662-ba16-5e3f7ce1803c None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:52.939438 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c35a55-8d28-4662-ba16-5e3f7ce1803c None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:52.939711 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c35a55-8d28-4662-ba16-5e3f7ce1803c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:52.960677 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d1c35a55-8d28-4662-ba16-5e3f7ce1803c None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:52.961944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d1c35a55-8d28-4662-ba16-5e3f7ce1803c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:52.970223 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1240/2485] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:43:52.974212 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c35a55-8d28-4662-ba16-5e3f7ce1803c None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:52.983142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c35a55-8d28-4662-ba16-5e3f7ce1803c None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:52.986071 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce08b9a-5a65-4e38-b5a2-fdfeb1c68e86 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:52.988239 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b91374-a6f7-4c29-ad30-ab482025dfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:52.988819 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b91374-a6f7-4c29-ad30-ab482025dfd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:52.989127 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b91374-a6f7-4c29-ad30-ab482025dfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:52.989398 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b91374-a6f7-4c29-ad30-ab482025dfd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:52.990043 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b91374-a6f7-4c29-ad30-ab482025dfd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:43:52.990341 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b91374-a6f7-4c29-ad30-ab482025dfd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:52.991052 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613ed80-a46c-4743-b4d4-cb8041d8da82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:52.991684 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613ed80-a46c-4743-b4d4-cb8041d8da82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:52.991969 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613ed80-a46c-4743-b4d4-cb8041d8da82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:52.992172 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b91374-a6f7-4c29-ad30-ab482025dfd7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:52.992280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613ed80-a46c-4743-b4d4-cb8041d8da82 admin admin] PATH_INFO: `/v3/users/a1a0bc0d72614fabba6d24fa88e7d8bd` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:52.993422 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6613ed80-a46c-4743-b4d4-cb8041d8da82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1a0bc0d72614fabba6d24fa88e7d8bd)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:52.995337 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6613ed80-a46c-4743-b4d4-cb8041d8da82 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:53.004263 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1241/2486] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:43:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 20 13:43:53.008452 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b91374-a6f7-4c29-ad30-ab482025dfd7 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:53.018351 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b91374-a6f7-4c29-ad30-ab482025dfd7 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:53.023554 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389037d9-df1e-44c7-a906-ef9cc12d2dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:53.024218 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389037d9-df1e-44c7-a906-ef9cc12d2dfa admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:53.024669 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389037d9-df1e-44c7-a906-ef9cc12d2dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:53.024789 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389037d9-df1e-44c7-a906-ef9cc12d2dfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:53.025679 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389037d9-df1e-44c7-a906-ef9cc12d2dfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:53.028117 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389037d9-df1e-44c7-a906-ef9cc12d2dfa admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:53.043265 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6613ed80-a46c-4743-b4d4-cb8041d8da82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1a0bc0d72614fabba6d24fa88e7d8bd'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:53.043935 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6613ed80-a46c-4743-b4d4-cb8041d8da82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1a0bc0d72614fabba6d24fa88e7d8bd'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:53.051559 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6613ed80-a46c-4743-b4d4-cb8041d8da82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1a0bc0d72614fabba6d24fa88e7d8bd'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:53.052313 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6613ed80-a46c-4743-b4d4-cb8041d8da82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1a0bc0d72614fabba6d24fa88e7d8bd'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:53.056029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1242/2487] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:43:53 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:43:53.059320 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-6613ed80-a46c-4743-b4d4-cb8041d8da82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1a0bc0d72614fabba6d24fa88e7d8bd'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:53.060073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389037d9-df1e-44c7-a906-ef9cc12d2dfa admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:53.061668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-389037d9-df1e-44c7-a906-ef9cc12d2dfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:53.077140 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1246/2488] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:43:52 2022] DELETE /identity/v3/users/a1a0bc0d72614fabba6d24fa88e7d8bd => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:43:53.082225 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ace54-5584-46d6-923d-b69ae3795645 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:53.082508 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ace54-5584-46d6-923d-b69ae3795645 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:53.082778 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ace54-5584-46d6-923d-b69ae3795645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:53.101031 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-403ace54-5584-46d6-923d-b69ae3795645 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:53.102213 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-403ace54-5584-46d6-923d-b69ae3795645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:53.127236 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389037d9-df1e-44c7-a906-ef9cc12d2dfa admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:53.164228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1247/2489] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:43:53.174160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ace54-5584-46d6-923d-b69ae3795645 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:53.175513 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-403ace54-5584-46d6-923d-b69ae3795645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:53.183117 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7b41a-db11-4f0c-a365-11cd8d08198f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:53.183725 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7b41a-db11-4f0c-a365-11cd8d08198f admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:53.184002 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7b41a-db11-4f0c-a365-11cd8d08198f admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:53.184273 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7b41a-db11-4f0c-a365-11cd8d08198f admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:53.185124 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7b41a-db11-4f0c-a365-11cd8d08198f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:53.187456 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7b41a-db11-4f0c-a365-11cd8d08198f admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:53.193374 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:53.215652 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1243/2490] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:43:53 2022] POST /identity/v3/users => generated 287 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:43:53.216938 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7b41a-db11-4f0c-a365-11cd8d08198f admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:53.227013 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7b41a-db11-4f0c-a365-11cd8d08198f admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:53.231757 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3cec0-93c3-4420-a123-3bafb3a30924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:53.232341 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3cec0-93c3-4420-a123-3bafb3a30924 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:53.232616 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3cec0-93c3-4420-a123-3bafb3a30924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:53.233033 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3cec0-93c3-4420-a123-3bafb3a30924 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:53.233670 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e3cec0-93c3-4420-a123-3bafb3a30924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:53.235813 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e3cec0-93c3-4420-a123-3bafb3a30924 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:53.237280 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ace54-5584-46d6-923d-b69ae3795645 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:53.241495 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74846ad0-ce43-46d9-b3c5-fe271d00331a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:53.242129 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74846ad0-ce43-46d9-b3c5-fe271d00331a service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:53.242409 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74846ad0-ce43-46d9-b3c5-fe271d00331a service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:53.242678 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74846ad0-ce43-46d9-b3c5-fe271d00331a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:53.245973 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1244/2491] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:43:53.251631 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3cec0-93c3-4420-a123-3bafb3a30924 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:53.252855 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74846ad0-ce43-46d9-b3c5-fe271d00331a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:53.254307 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74846ad0-ce43-46d9-b3c5-fe271d00331a service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:53.261373 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3cec0-93c3-4420-a123-3bafb3a30924 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:53.264444 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1248/2492] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:43:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:43:53.267600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e62ef5-a679-4a61-96ae-a455baa85cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:53.268207 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e62ef5-a679-4a61-96ae-a455baa85cac admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:53.268488 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e62ef5-a679-4a61-96ae-a455baa85cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:53.268784 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e62ef5-a679-4a61-96ae-a455baa85cac admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:53.269455 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e62ef5-a679-4a61-96ae-a455baa85cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:53.271181 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e62ef5-a679-4a61-96ae-a455baa85cac admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:53.283414 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1245/2493] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:43:53.287452 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74846ad0-ce43-46d9-b3c5-fe271d00331a service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:53.297648 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74846ad0-ce43-46d9-b3c5-fe271d00331a service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:53.303793 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d0bf6-1584-4e3c-a57a-713e20c354de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:53.304889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d0bf6-1584-4e3c-a57a-713e20c354de admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:53.305248 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d0bf6-1584-4e3c-a57a-713e20c354de admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:53.305531 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d0bf6-1584-4e3c-a57a-713e20c354de admin admin] PATH_INFO: `/v3/projects/aad84947a40f4e7eada99082ee91ab34/users/c4f1ca4842b54591ae328adb0a8e851b/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:53.325015 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d0bf6-1584-4e3c-a57a-713e20c354de admin admin] RBAC: Authorizing `identity:create_grant(project_id=aad84947a40f4e7eada99082ee91ab34, user_id=c4f1ca4842b54591ae328adb0a8e851b, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:53.328198 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d0bf6-1584-4e3c-a57a-713e20c354de admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:53.361168 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1249/2494] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:43:53 2022] PUT /identity/v3/projects/aad84947a40f4e7eada99082ee91ab34/users/c4f1ca4842b54591ae328adb0a8e851b/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:43:53.372238 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0ab02-bdc3-4f49-a665-6b15b1baa72f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:53.372667 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0ab02-bdc3-4f49-a665-6b15b1baa72f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:53.372939 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0ab02-bdc3-4f49-a665-6b15b1baa72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:53.407054 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-43b0ab02-bdc3-4f49-a665-6b15b1baa72f None None] MFA Rules not processed for user `c4f1ca4842b54591ae328adb0a8e851b`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:53.408439 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-43b0ab02-bdc3-4f49-a665-6b15b1baa72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:53.431739 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:53.480433 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1246/2495] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:53.484467 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d0bf6-1584-4e3c-a57a-713e20c354de admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:53.560201 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d0bf6-1584-4e3c-a57a-713e20c354de admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:53.562122 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0ab02-bdc3-4f49-a665-6b15b1baa72f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:53.564397 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-43b0ab02-bdc3-4f49-a665-6b15b1baa72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:53.566166 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f0d3f-7b57-4190-b05e-2d212b1afc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:53.566720 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f0d3f-7b57-4190-b05e-2d212b1afc07 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:53.567004 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f0d3f-7b57-4190-b05e-2d212b1afc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:53.567385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f0d3f-7b57-4190-b05e-2d212b1afc07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:53.567894 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f0d3f-7b57-4190-b05e-2d212b1afc07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:53.570046 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f0d3f-7b57-4190-b05e-2d212b1afc07 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:53.580487 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0ab02-bdc3-4f49-a665-6b15b1baa72f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:53.586773 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83edfc2-2bcd-42dc-82c3-9f348087cefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:53.587499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83edfc2-2bcd-42dc-82c3-9f348087cefd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:53.587899 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83edfc2-2bcd-42dc-82c3-9f348087cefd admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:53.588182 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83edfc2-2bcd-42dc-82c3-9f348087cefd admin admin] PATH_INFO: `/v3/projects/b947d0325b7245dda895da87fc8a28ec` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:53.589922 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83edfc2-2bcd-42dc-82c3-9f348087cefd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b947d0325b7245dda895da87fc8a28ec)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:53.592135 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83edfc2-2bcd-42dc-82c3-9f348087cefd admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:53.602116 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1250/2496] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:43:53 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:43:53.606087 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f0d3f-7b57-4190-b05e-2d212b1afc07 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:53.647073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f83edfc2-2bcd-42dc-82c3-9f348087cefd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b947d0325b7245dda895da87fc8a28ec', 'request_id': 'req-f83edfc2-2bcd-42dc-82c3-9f348087cefd'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:53.654243 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f83edfc2-2bcd-42dc-82c3-9f348087cefd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b947d0325b7245dda895da87fc8a28ec', 'request_id': 'req-f83edfc2-2bcd-42dc-82c3-9f348087cefd'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:53.662629 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f83edfc2-2bcd-42dc-82c3-9f348087cefd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b947d0325b7245dda895da87fc8a28ec', 'request_id': 'req-f83edfc2-2bcd-42dc-82c3-9f348087cefd'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:53.674944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f83edfc2-2bcd-42dc-82c3-9f348087cefd admin admin] The token cache is being invalidate because project b947d0325b7245dda895da87fc8a28ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:43:53.675245 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f83edfc2-2bcd-42dc-82c3-9f348087cefd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:53.675903 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-f83edfc2-2bcd-42dc-82c3-9f348087cefd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:53.676298 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f0d3f-7b57-4190-b05e-2d212b1afc07 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:53.677527 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1247/2497] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:43:53 2022] DELETE /identity/v3/projects/b947d0325b7245dda895da87fc8a28ec => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:53.683185 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2e5c2-17bd-4101-9b86-fe2b17b4e290 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:53.683556 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2e5c2-17bd-4101-9b86-fe2b17b4e290 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:53.683847 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2e5c2-17bd-4101-9b86-fe2b17b4e290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:53.702344 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:53.705476 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-d1d2e5c2-17bd-4101-9b86-fe2b17b4e290 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:53.706701 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d1d2e5c2-17bd-4101-9b86-fe2b17b4e290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:53.744199 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cc0e1-4cb6-46c9-9321-58767406e371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:53.744835 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cc0e1-4cb6-46c9-9321-58767406e371 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:53.745141 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cc0e1-4cb6-46c9-9321-58767406e371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:53.745415 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cc0e1-4cb6-46c9-9321-58767406e371 admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:53.746199 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94cc0e1-4cb6-46c9-9321-58767406e371 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:53.748137 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94cc0e1-4cb6-46c9-9321-58767406e371 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:53.773658 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1251/2498] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:43:53 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:43:53.773870 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1248/2499] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:43:53.775141 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cc0e1-4cb6-46c9-9321-58767406e371 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:53.781167 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-a94cc0e1-4cb6-46c9-9321-58767406e371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:53.782947 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592d599-dfca-448c-8ae5-54e900bd4b8f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:53.783234 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592d599-dfca-448c-8ae5-54e900bd4b8f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:53.783505 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592d599-dfca-448c-8ae5-54e900bd4b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:53.794257 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cc0e1-4cb6-46c9-9321-58767406e371 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:53.799764 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8af36-05ad-429d-886d-e8760ead833e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:53.800359 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8af36-05ad-429d-886d-e8760ead833e admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:53.800641 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8af36-05ad-429d-886d-e8760ead833e admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:53.800935 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8af36-05ad-429d-886d-e8760ead833e admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:53.801595 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8af36-05ad-429d-886d-e8760ead833e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:53.802613 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-9592d599-dfca-448c-8ae5-54e900bd4b8f None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:53.803166 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c8af36-05ad-429d-886d-e8760ead833e admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:53.803772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-9592d599-dfca-448c-8ae5-54e900bd4b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:53.811912 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1249/2500] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:43:53.814520 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1252/2501] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:43:53.816884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592d599-dfca-448c-8ae5-54e900bd4b8f None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:53.818172 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8af36-05ad-429d-886d-e8760ead833e admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:53.826619 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592d599-dfca-448c-8ae5-54e900bd4b8f None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:53.827686 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8af36-05ad-429d-886d-e8760ead833e admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:53.832477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f10e70-4d0f-4322-9ff9-9a7c4ddf73d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:53.832961 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c5865-d85d-46bc-bc79-a14b63c98420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:53.833093 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f10e70-4d0f-4322-9ff9-9a7c4ddf73d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:53.833372 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f10e70-4d0f-4322-9ff9-9a7c4ddf73d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:53.833552 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c5865-d85d-46bc-bc79-a14b63c98420 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:53.833668 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f10e70-4d0f-4322-9ff9-9a7c4ddf73d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:53.833856 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c5865-d85d-46bc-bc79-a14b63c98420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:53.834115 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c5865-d85d-46bc-bc79-a14b63c98420 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:53.834275 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f10e70-4d0f-4322-9ff9-9a7c4ddf73d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:43:53.834579 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f10e70-4d0f-4322-9ff9-9a7c4ddf73d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:53.834994 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c5865-d85d-46bc-bc79-a14b63c98420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:53.836171 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f10e70-4d0f-4322-9ff9-9a7c4ddf73d3 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:53.836736 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c5865-d85d-46bc-bc79-a14b63c98420 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:53.850235 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1250/2502] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:43:53.851165 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1253/2503] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:43:53.854231 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f10e70-4d0f-4322-9ff9-9a7c4ddf73d3 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:53.854401 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c5865-d85d-46bc-bc79-a14b63c98420 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:53.864099 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f10e70-4d0f-4322-9ff9-9a7c4ddf73d3 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:53.865001 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c5865-d85d-46bc-bc79-a14b63c98420 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:53.869196 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b036cd-c2ec-43c6-90bc-f31e4ddf42a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:53.869849 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b036cd-c2ec-43c6-90bc-f31e4ddf42a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:53.869964 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6cae82-bfde-4cfc-96b9-ccc937f1542b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:53.870161 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b036cd-c2ec-43c6-90bc-f31e4ddf42a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:53.870440 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b036cd-c2ec-43c6-90bc-f31e4ddf42a5 admin admin] PATH_INFO: `/v3/projects/9fcc02cc00764387bc20193afd67492c/users/030267a47ab24bd2ae40aa6d2954f4c8/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:53.870540 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cae82-bfde-4cfc-96b9-ccc937f1542b admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:53.870771 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cae82-bfde-4cfc-96b9-ccc937f1542b admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:53.871044 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cae82-bfde-4cfc-96b9-ccc937f1542b admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:53.871680 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6cae82-bfde-4cfc-96b9-ccc937f1542b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:53.874652 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6cae82-bfde-4cfc-96b9-ccc937f1542b admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:53.886518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b036cd-c2ec-43c6-90bc-f31e4ddf42a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fcc02cc00764387bc20193afd67492c, user_id=030267a47ab24bd2ae40aa6d2954f4c8, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:53.889192 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b036cd-c2ec-43c6-90bc-f31e4ddf42a5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:53.904294 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1251/2504] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:43:53 2022] PUT /identity/v3/projects/9fcc02cc00764387bc20193afd67492c/users/030267a47ab24bd2ae40aa6d2954f4c8/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:43:53.904457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1254/2505] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:43:53 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:43:53.908740 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6cae82-bfde-4cfc-96b9-ccc937f1542b admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:53.917394 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210dc71-fdfc-41a8-a88e-33297dc4729d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:53.917678 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210dc71-fdfc-41a8-a88e-33297dc4729d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:53.917946 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210dc71-fdfc-41a8-a88e-33297dc4729d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:53.925454 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:53.950411 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c210dc71-fdfc-41a8-a88e-33297dc4729d None None] MFA Rules not processed for user `030267a47ab24bd2ae40aa6d2954f4c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:53.951653 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c210dc71-fdfc-41a8-a88e-33297dc4729d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:53.972923 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6cae82-bfde-4cfc-96b9-ccc937f1542b admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:53.977641 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c95914-49bd-4353-8ad3-76dba67c4a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:53.978251 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c95914-49bd-4353-8ad3-76dba67c4a1c admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:53.978534 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c95914-49bd-4353-8ad3-76dba67c4a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:53.978809 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c95914-49bd-4353-8ad3-76dba67c4a1c admin admin] PATH_INFO: `/v3/users` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:53.979461 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c95914-49bd-4353-8ad3-76dba67c4a1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:53.981540 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c95914-49bd-4353-8ad3-76dba67c4a1c admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:54.004931 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1255/2506] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:43:53 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:43:54.008707 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c95914-49bd-4353-8ad3-76dba67c4a1c admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:54.017641 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1252/2507] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:54.021405 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c95914-49bd-4353-8ad3-76dba67c4a1c admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:54.022093 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c210dc71-fdfc-41a8-a88e-33297dc4729d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:54.023692 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced43ba8-d827-4f02-9510-22408c1ea71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:54.024291 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced43ba8-d827-4f02-9510-22408c1ea71d admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:54.024574 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced43ba8-d827-4f02-9510-22408c1ea71d admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:54.024855 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced43ba8-d827-4f02-9510-22408c1ea71d admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:54.025527 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced43ba8-d827-4f02-9510-22408c1ea71d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:54.027289 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced43ba8-d827-4f02-9510-22408c1ea71d admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:54.032536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c210dc71-fdfc-41a8-a88e-33297dc4729d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:54.037053 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68596de6-aa4c-4704-ac29-1e4c0afdc188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:54.037639 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68596de6-aa4c-4704-ac29-1e4c0afdc188 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:54.037916 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68596de6-aa4c-4704-ac29-1e4c0afdc188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:54.038034 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1256/2508] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:43:54.038190 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68596de6-aa4c-4704-ac29-1e4c0afdc188 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:54.038810 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68596de6-aa4c-4704-ac29-1e4c0afdc188 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:54.040723 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68596de6-aa4c-4704-ac29-1e4c0afdc188 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:54.041931 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced43ba8-d827-4f02-9510-22408c1ea71d admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:54.052016 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced43ba8-d827-4f02-9510-22408c1ea71d admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:54.056839 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d45eaf-feb0-479c-8643-c65f83f93e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:54.057464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d45eaf-feb0-479c-8643-c65f83f93e0f admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:54.057741 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d45eaf-feb0-479c-8643-c65f83f93e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:54.058012 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d45eaf-feb0-479c-8643-c65f83f93e0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:54.058644 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d45eaf-feb0-479c-8643-c65f83f93e0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:54.061339 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d45eaf-feb0-479c-8643-c65f83f93e0f admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:54.069022 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1253/2509] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:43:54 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:43:54.072788 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68596de6-aa4c-4704-ac29-1e4c0afdc188 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:54.074571 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1257/2510] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:43:54.078368 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d45eaf-feb0-479c-8643-c65f83f93e0f admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:54.088545 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d45eaf-feb0-479c-8643-c65f83f93e0f admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:54.127106 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:54.138062 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68596de6-aa4c-4704-ac29-1e4c0afdc188 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:54.143399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ad089-4819-4f2f-8ed9-90d582eb0098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:54.144005 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ad089-4819-4f2f-8ed9-90d582eb0098 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:54.144291 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ad089-4819-4f2f-8ed9-90d582eb0098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:54.144557 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ad089-4819-4f2f-8ed9-90d582eb0098 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:54.145224 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167ad089-4819-4f2f-8ed9-90d582eb0098 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:54.147160 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167ad089-4819-4f2f-8ed9-90d582eb0098 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:54.147304 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122cbf9f-8564-4558-b4ee-230ec93167e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:54.147919 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122cbf9f-8564-4558-b4ee-230ec93167e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:54.148209 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122cbf9f-8564-4558-b4ee-230ec93167e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:54.148595 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122cbf9f-8564-4558-b4ee-230ec93167e7 admin admin] PATH_INFO: `/v3/projects/29dff2585070499ea1cdaa39016417d0/users/69c1e55e00094866897bc509ae65c6e5/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:54.164716 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122cbf9f-8564-4558-b4ee-230ec93167e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29dff2585070499ea1cdaa39016417d0, user_id=69c1e55e00094866897bc509ae65c6e5, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:54.167318 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122cbf9f-8564-4558-b4ee-230ec93167e7 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:54.170884 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1254/2511] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:43:54 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:43:54.173483 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ad089-4819-4f2f-8ed9-90d582eb0098 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:54.181692 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1258/2512] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:43:54 2022] PUT /identity/v3/projects/29dff2585070499ea1cdaa39016417d0/users/69c1e55e00094866897bc509ae65c6e5/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:43:54.190075 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ad089-4819-4f2f-8ed9-90d582eb0098 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:54.192363 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81391676-7795-41e4-a27a-950df09f7d8d None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:54.192649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81391676-7795-41e4-a27a-950df09f7d8d None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:54.193030 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81391676-7795-41e4-a27a-950df09f7d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:54.223110 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-81391676-7795-41e4-a27a-950df09f7d8d None None] MFA Rules not processed for user `69c1e55e00094866897bc509ae65c6e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:54.224332 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-81391676-7795-41e4-a27a-950df09f7d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:54.250263 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047089e-f080-458d-8069-c89fad2bbf69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:54.251077 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047089e-f080-458d-8069-c89fad2bbf69 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:54.251369 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047089e-f080-458d-8069-c89fad2bbf69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:54.251644 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047089e-f080-458d-8069-c89fad2bbf69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:54.252293 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047089e-f080-458d-8069-c89fad2bbf69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:54.253915 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047089e-f080-458d-8069-c89fad2bbf69 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:54.255217 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:54.268122 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1255/2513] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:43:54.271911 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047089e-f080-458d-8069-c89fad2bbf69 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:54.281574 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047089e-f080-458d-8069-c89fad2bbf69 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:54.286566 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1201c-1db3-42f0-bb20-66267f1c1635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:54.287230 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1201c-1db3-42f0-bb20-66267f1c1635 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:54.287508 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1201c-1db3-42f0-bb20-66267f1c1635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:54.287781 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1201c-1db3-42f0-bb20-66267f1c1635 admin admin] PATH_INFO: `/v3/projects/4aca3222198e493aad90346ddceffef9/users/934a4e45bb9b47158ddb2546c7df57de/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:54.290212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1259/2514] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:43:54.300915 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81391676-7795-41e4-a27a-950df09f7d8d None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:54.302275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-81391676-7795-41e4-a27a-950df09f7d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:54.302562 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc1201c-1db3-42f0-bb20-66267f1c1635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aca3222198e493aad90346ddceffef9, user_id=934a4e45bb9b47158ddb2546c7df57de, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:54.305343 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc1201c-1db3-42f0-bb20-66267f1c1635 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:54.320361 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1256/2515] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:43:54 2022] PUT /identity/v3/projects/4aca3222198e493aad90346ddceffef9/users/934a4e45bb9b47158ddb2546c7df57de/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:43:54.324244 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1201c-1db3-42f0-bb20-66267f1c1635 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:54.359929 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81391676-7795-41e4-a27a-950df09f7d8d None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:54.385508 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1201c-1db3-42f0-bb20-66267f1c1635 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:54.390242 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f183c6b-620a-453d-ade4-84d5b5f15c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:54.390845 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f183c6b-620a-453d-ade4-84d5b5f15c8e admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:54.391133 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f183c6b-620a-453d-ade4-84d5b5f15c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:54.392053 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f183c6b-620a-453d-ade4-84d5b5f15c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:54.392700 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f183c6b-620a-453d-ade4-84d5b5f15c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:54.394328 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f183c6b-620a-453d-ade4-84d5b5f15c8e admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:54.404164 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1257/2516] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:43:54.408957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f183c6b-620a-453d-ade4-84d5b5f15c8e admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:54.411480 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f1cb6-1dc8-4bc9-abf5-b5afe2736329 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:54.412300 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f1cb6-1dc8-4bc9-abf5-b5afe2736329 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:54.413383 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f1cb6-1dc8-4bc9-abf5-b5afe2736329 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:54.413935 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f1cb6-1dc8-4bc9-abf5-b5afe2736329 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:54.418977 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f183c6b-620a-453d-ade4-84d5b5f15c8e admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:54.424172 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12f2b4-6b60-4062-9397-36f86003f9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:54.424827 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12f2b4-6b60-4062-9397-36f86003f9d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:54.425188 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12f2b4-6b60-4062-9397-36f86003f9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:54.425468 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12f2b4-6b60-4062-9397-36f86003f9d5 admin admin] PATH_INFO: `/v3/projects/4aca3222198e493aad90346ddceffef9/users/934a4e45bb9b47158ddb2546c7df57de/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:54.427402 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12f2b4-6b60-4062-9397-36f86003f9d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aca3222198e493aad90346ddceffef9, user_id=934a4e45bb9b47158ddb2546c7df57de, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:54.430083 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12f2b4-6b60-4062-9397-36f86003f9d5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:54.440702 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:54.444430 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1258/2517] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:43:54 2022] PUT /identity/v3/projects/4aca3222198e493aad90346ddceffef9/users/934a4e45bb9b47158ddb2546c7df57de/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:43:54.456993 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185fb6f-a782-40e1-9b19-1fd82183594d None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:54.457357 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185fb6f-a782-40e1-9b19-1fd82183594d None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:54.457646 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185fb6f-a782-40e1-9b19-1fd82183594d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:54.486249 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f1cb6-1dc8-4bc9-abf5-b5afe2736329 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:54.487841 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f1cb6-1dc8-4bc9-abf5-b5afe2736329 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:54.495581 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-2185fb6f-a782-40e1-9b19-1fd82183594d None None] MFA Rules not processed for user `934a4e45bb9b47158ddb2546c7df57de`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:54.496318 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2185fb6f-a782-40e1-9b19-1fd82183594d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:54.499718 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1260/2518] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:43:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:43:54.585049 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1259/2519] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:54.597316 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f1cb6-1dc8-4bc9-abf5-b5afe2736329 service cinder] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:54.598629 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-114f1cb6-1dc8-4bc9-abf5-b5afe2736329 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:54.611124 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f1cb6-1dc8-4bc9-abf5-b5afe2736329 service cinder] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:54.671434 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5bbfa4-7813-46dd-a214-8df321c84336 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:54.671767 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5bbfa4-7813-46dd-a214-8df321c84336 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:54.672050 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5bbfa4-7813-46dd-a214-8df321c84336 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:54.672291 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5bbfa4-7813-46dd-a214-8df321c84336 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:54.675370 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-fd5bbfa4-7813-46dd-a214-8df321c84336 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:54.740408 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5bbfa4-7813-46dd-a214-8df321c84336 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:54.742469 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5bbfa4-7813-46dd-a214-8df321c84336 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:54.754245 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1261/2520] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:43:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:43:55.583287 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185fb6f-a782-40e1-9b19-1fd82183594d None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:55.585073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-2185fb6f-a782-40e1-9b19-1fd82183594d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:55.628252 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:55.650874 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185fb6f-a782-40e1-9b19-1fd82183594d None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:55.655299 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9aa07b-a6eb-423f-a642-35aff2451e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:55.655938 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9aa07b-a6eb-423f-a642-35aff2451e00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:55.656215 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9aa07b-a6eb-423f-a642-35aff2451e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:55.656484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9aa07b-a6eb-423f-a642-35aff2451e00 admin admin] PATH_INFO: `/v3/users/c78507eb11ef44609970b6746c6638e9` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:55.658451 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9aa07b-a6eb-423f-a642-35aff2451e00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c78507eb11ef44609970b6746c6638e9)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:55.660352 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9aa07b-a6eb-423f-a642-35aff2451e00 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:55.705493 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-db9aa07b-a6eb-423f-a642-35aff2451e00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c78507eb11ef44609970b6746c6638e9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:55.706128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-db9aa07b-a6eb-423f-a642-35aff2451e00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c78507eb11ef44609970b6746c6638e9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:55.713247 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-db9aa07b-a6eb-423f-a642-35aff2451e00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c78507eb11ef44609970b6746c6638e9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:55.721020 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-db9aa07b-a6eb-423f-a642-35aff2451e00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c78507eb11ef44609970b6746c6638e9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:55.721860 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-db9aa07b-a6eb-423f-a642-35aff2451e00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c78507eb11ef44609970b6746c6638e9'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:55.739470 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1260/2521] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:43:55 2022] DELETE /identity/v3/users/c78507eb11ef44609970b6746c6638e9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:55.744508 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d5c38-0694-4e49-b336-bd705363aadc None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:55.745014 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d5c38-0694-4e49-b336-bd705363aadc None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:55.745204 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d5c38-0694-4e49-b336-bd705363aadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:55.764822 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-d29d5c38-0694-4e49-b336-bd705363aadc None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:55.765974 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-d29d5c38-0694-4e49-b336-bd705363aadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:55.822508 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1262/2522] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:55.832472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9aa07b-a6eb-423f-a642-35aff2451e00 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:55.833901 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-db9aa07b-a6eb-423f-a642-35aff2451e00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:55.893061 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9aa07b-a6eb-423f-a642-35aff2451e00 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:55.896716 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b39cd-5c9f-4c89-8456-604204817b3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:55.897351 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b39cd-5c9f-4c89-8456-604204817b3b service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:55.897654 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b39cd-5c9f-4c89-8456-604204817b3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:55.897891 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b39cd-5c9f-4c89-8456-604204817b3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:55.908143 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b39cd-5c9f-4c89-8456-604204817b3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:55.909594 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b39cd-5c9f-4c89-8456-604204817b3b service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:55.919298 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1261/2523] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:43:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:43:55.920959 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430036cf-df1a-43dc-aa12-72fb0d9dadca None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:55.921272 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430036cf-df1a-43dc-aa12-72fb0d9dadca None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:55.921537 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430036cf-df1a-43dc-aa12-72fb0d9dadca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:55.941352 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-430036cf-df1a-43dc-aa12-72fb0d9dadca None None] MFA Rules not processed for user `69c1e55e00094866897bc509ae65c6e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:55.942566 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-430036cf-df1a-43dc-aa12-72fb0d9dadca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:55.954442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=100 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:56.002414 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1263/2524] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:43:56.284131 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b39cd-5c9f-4c89-8456-604204817b3b service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:56.285647 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-6b0b39cd-5c9f-4c89-8456-604204817b3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:56.344563 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b39cd-5c9f-4c89-8456-604204817b3b service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:56.348632 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d0f212-0d54-4fb4-8d15-ca80b70e350f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:56.349300 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0f212-0d54-4fb4-8d15-ca80b70e350f service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:56.349574 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0f212-0d54-4fb4-8d15-ca80b70e350f service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:56.349842 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0f212-0d54-4fb4-8d15-ca80b70e350f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:56.351540 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-d7d0f212-0d54-4fb4-8d15-ca80b70e350f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:56.366957 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:56.419892 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d0f212-0d54-4fb4-8d15-ca80b70e350f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:56.421398 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d0f212-0d54-4fb4-8d15-ca80b70e350f service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:56.433122 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1262/2525] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:43:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:43:56.437183 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430036cf-df1a-43dc-aa12-72fb0d9dadca None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:56.439875 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-430036cf-df1a-43dc-aa12-72fb0d9dadca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:56.464431 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430036cf-df1a-43dc-aa12-72fb0d9dadca None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:56.481890 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a80b6d-b680-4d37-b964-fecdc7a75f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:56.482539 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a80b6d-b680-4d37-b964-fecdc7a75f38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:56.482817 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a80b6d-b680-4d37-b964-fecdc7a75f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:56.483078 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a80b6d-b680-4d37-b964-fecdc7a75f38 admin admin] PATH_INFO: `/v3/projects/d2f47c67111a49e184409ce751039485` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:56.484522 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a80b6d-b680-4d37-b964-fecdc7a75f38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2f47c67111a49e184409ce751039485)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:56.487136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a80b6d-b680-4d37-b964-fecdc7a75f38 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:56.559721 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c2a80b6d-b680-4d37-b964-fecdc7a75f38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2f47c67111a49e184409ce751039485', 'request_id': 'req-c2a80b6d-b680-4d37-b964-fecdc7a75f38'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:56.570674 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c2a80b6d-b680-4d37-b964-fecdc7a75f38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2f47c67111a49e184409ce751039485', 'request_id': 'req-c2a80b6d-b680-4d37-b964-fecdc7a75f38'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:56.585482 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c2a80b6d-b680-4d37-b964-fecdc7a75f38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2f47c67111a49e184409ce751039485', 'request_id': 'req-c2a80b6d-b680-4d37-b964-fecdc7a75f38'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:56.600980 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c2a80b6d-b680-4d37-b964-fecdc7a75f38 admin admin] The token cache is being invalidate because project d2f47c67111a49e184409ce751039485 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:43:56.601234 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c2a80b6d-b680-4d37-b964-fecdc7a75f38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:56.602194 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-c2a80b6d-b680-4d37-b964-fecdc7a75f38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:43:56.604719 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1264/2526] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:43:56 2022] DELETE /identity/v3/projects/d2f47c67111a49e184409ce751039485 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:56.610479 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dffb5-028a-41e9-afcc-487e1180273e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:56.610982 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dffb5-028a-41e9-afcc-487e1180273e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:56.611122 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dffb5-028a-41e9-afcc-487e1180273e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:56.638669 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-cc8dffb5-028a-41e9-afcc-487e1180273e None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:56.640857 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-cc8dffb5-028a-41e9-afcc-487e1180273e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:56.723500 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1263/2527] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:56.731622 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d8f4f-02b7-45fd-b6a2-898078e6b553 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:56.731979 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d8f4f-02b7-45fd-b6a2-898078e6b553 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:56.732340 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d8f4f-02b7-45fd-b6a2-898078e6b553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:56.755631 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-7c7d8f4f-02b7-45fd-b6a2-898078e6b553 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:56.756955 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-7c7d8f4f-02b7-45fd-b6a2-898078e6b553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:56.765663 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1265/2528] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:43:56.770502 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8dffb5-028a-41e9-afcc-487e1180273e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:56.781536 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8dffb5-028a-41e9-afcc-487e1180273e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:56.788001 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4501cff3-4786-488d-90ed-fd76c711e5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:56.788637 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501cff3-4786-488d-90ed-fd76c711e5b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:56.788939 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501cff3-4786-488d-90ed-fd76c711e5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:56.789215 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501cff3-4786-488d-90ed-fd76c711e5b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:56.789915 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4501cff3-4786-488d-90ed-fd76c711e5b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:43:56.790160 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4501cff3-4786-488d-90ed-fd76c711e5b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:56.791770 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4501cff3-4786-488d-90ed-fd76c711e5b7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:56.809374 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1264/2529] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:43:56.810697 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d8f4f-02b7-45fd-b6a2-898078e6b553 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:56.821227 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d8f4f-02b7-45fd-b6a2-898078e6b553 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:56.827162 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614e7e2-5553-47ea-bd9d-6a719067cb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:56.827795 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614e7e2-5553-47ea-bd9d-6a719067cb89 admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:56.828071 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614e7e2-5553-47ea-bd9d-6a719067cb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:56.828467 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614e7e2-5553-47ea-bd9d-6a719067cb89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:56.830027 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9614e7e2-5553-47ea-bd9d-6a719067cb89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:56.831967 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9614e7e2-5553-47ea-bd9d-6a719067cb89 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:56.863473 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1266/2530] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:43:56 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:43:56.868019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4501cff3-4786-488d-90ed-fd76c711e5b7 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:56.879789 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4501cff3-4786-488d-90ed-fd76c711e5b7 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:56.960971 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb16f2-8ed8-4340-b9fe-5cc35728110d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:56.961664 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb16f2-8ed8-4340-b9fe-5cc35728110d admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:56.962560 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb16f2-8ed8-4340-b9fe-5cc35728110d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:56.962560 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb16f2-8ed8-4340-b9fe-5cc35728110d admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:56.963567 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb16f2-8ed8-4340-b9fe-5cc35728110d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:56.965632 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb16f2-8ed8-4340-b9fe-5cc35728110d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:56.993101 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1265/2531] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:43:56 2022] POST /identity/v3/users => generated 286 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:43:56.997302 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614e7e2-5553-47ea-bd9d-6a719067cb89 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:57.022583 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614e7e2-5553-47ea-bd9d-6a719067cb89 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:57.030428 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f4279-6b65-4ea8-b6c9-00d1644d4016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:57.031139 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f4279-6b65-4ea8-b6c9-00d1644d4016 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:57.031442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f4279-6b65-4ea8-b6c9-00d1644d4016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:57.031733 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f4279-6b65-4ea8-b6c9-00d1644d4016 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:57.032691 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01f4279-6b65-4ea8-b6c9-00d1644d4016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:57.034476 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01f4279-6b65-4ea8-b6c9-00d1644d4016 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:57.050417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1267/2532] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:43:57.051570 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb16f2-8ed8-4340-b9fe-5cc35728110d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:57.061532 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb16f2-8ed8-4340-b9fe-5cc35728110d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:57.067231 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58bee4-55eb-4dbd-a51b-3f95d1f06eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:57.067961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58bee4-55eb-4dbd-a51b-3f95d1f06eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:57.068372 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58bee4-55eb-4dbd-a51b-3f95d1f06eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:57.068750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58bee4-55eb-4dbd-a51b-3f95d1f06eb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:57.069626 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58bee4-55eb-4dbd-a51b-3f95d1f06eb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:57.071610 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58bee4-55eb-4dbd-a51b-3f95d1f06eb9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:57.086036 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1266/2533] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:43:57.094953 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f4279-6b65-4ea8-b6c9-00d1644d4016 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:57.106740 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f4279-6b65-4ea8-b6c9-00d1644d4016 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:57.112156 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d96670-55a7-4215-9e63-e07e25e4fd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:57.112848 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d96670-55a7-4215-9e63-e07e25e4fd42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:57.113157 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d96670-55a7-4215-9e63-e07e25e4fd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:57.113612 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d96670-55a7-4215-9e63-e07e25e4fd42 admin admin] PATH_INFO: `/v3/projects/2de4676254c14b55aa2b7765255c69c9/users/7f7a6b4a92ff4232843913a1dbd7f1c4/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:57.115707 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58bee4-55eb-4dbd-a51b-3f95d1f06eb9 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:57.117776 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-5f58bee4-55eb-4dbd-a51b-3f95d1f06eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:57.130670 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d96670-55a7-4215-9e63-e07e25e4fd42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2de4676254c14b55aa2b7765255c69c9, user_id=7f7a6b4a92ff4232843913a1dbd7f1c4, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:57.133796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d96670-55a7-4215-9e63-e07e25e4fd42 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:57.149337 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1268/2534] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:43:57 2022] PUT /identity/v3/projects/2de4676254c14b55aa2b7765255c69c9/users/7f7a6b4a92ff4232843913a1dbd7f1c4/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:43:57.150535 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:57.154447 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d65b9-bbc4-4255-9df3-770f29028a91 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:57.154740 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d65b9-bbc4-4255-9df3-770f29028a91 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:57.155068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d65b9-bbc4-4255-9df3-770f29028a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:57.180275 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-5b4d65b9-bbc4-4255-9df3-770f29028a91 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:43:57.181673 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-5b4d65b9-bbc4-4255-9df3-770f29028a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:57.208727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58bee4-55eb-4dbd-a51b-3f95d1f06eb9 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:57.224980 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:43:57.248627 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1269/2535] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:43:57.249720 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d65b9-bbc4-4255-9df3-770f29028a91 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:57.251665 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-5b4d65b9-bbc4-4255-9df3-770f29028a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:57.259818 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b32f3-1c39-4ed4-af95-61c983ff749e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:57.260425 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b32f3-1c39-4ed4-af95-61c983ff749e service cinder] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:57.260702 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b32f3-1c39-4ed4-af95-61c983ff749e service cinder] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:57.261002 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b32f3-1c39-4ed4-af95-61c983ff749e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:57.262765 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-500b32f3-1c39-4ed4-af95-61c983ff749e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:57.331699 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b32f3-1c39-4ed4-af95-61c983ff749e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:57.333698 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b32f3-1c39-4ed4-af95-61c983ff749e service cinder] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:57.343180 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1267/2536] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:43:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:43:57.344051 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b32f3-1c39-4ed4-af95-61c983ff749e service cinder] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:43:57.345885 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-500b32f3-1c39-4ed4-af95-61c983ff749e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:57.348896 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d65b9-bbc4-4255-9df3-770f29028a91 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:57.354085 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10859a-8c5e-4840-a334-72f4478355f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2440594934b64621a01c441133778155', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:57.354683 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10859a-8c5e-4840-a334-72f4478355f2 service placement] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:57.355125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10859a-8c5e-4840-a334-72f4478355f2 service placement] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:57.356747 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10859a-8c5e-4840-a334-72f4478355f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:57.357834 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-0d10859a-8c5e-4840-a334-72f4478355f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:43:57.370380 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10859a-8c5e-4840-a334-72f4478355f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:57.371966 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10859a-8c5e-4840-a334-72f4478355f2 service placement] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:57.384360 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1270/2537] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:43:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:43:57.410092 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b32f3-1c39-4ed4-af95-61c983ff749e service cinder] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:43:57.413753 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb6964-6e53-4d2b-a56a-502d84e17a76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:43:57.414395 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb6964-6e53-4d2b-a56a-502d84e17a76 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:43:57.414711 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb6964-6e53-4d2b-a56a-502d84e17a76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:43:57.415007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb6964-6e53-4d2b-a56a-502d84e17a76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:43:57.425863 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb6964-6e53-4d2b-a56a-502d84e17a76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:43:57.427308 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb6964-6e53-4d2b-a56a-502d84e17a76 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:43:57.439379 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1268/2538] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:43:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:44:01.624391 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10859a-8c5e-4840-a334-72f4478355f2 service placement] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:01.625914 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-0d10859a-8c5e-4840-a334-72f4478355f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:01.646384 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10859a-8c5e-4840-a334-72f4478355f2 service placement] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:01.651134 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79d016-c925-4ccf-adc9-c8582026edc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:01.651835 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79d016-c925-4ccf-adc9-c8582026edc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:01.652178 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79d016-c925-4ccf-adc9-c8582026edc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:01.652514 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79d016-c925-4ccf-adc9-c8582026edc6 admin admin] PATH_INFO: `/v3/users/c4f1ca4842b54591ae328adb0a8e851b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:01.653736 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b79d016-c925-4ccf-adc9-c8582026edc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4f1ca4842b54591ae328adb0a8e851b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:01.655639 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b79d016-c925-4ccf-adc9-c8582026edc6 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:01.722538 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4b79d016-c925-4ccf-adc9-c8582026edc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4f1ca4842b54591ae328adb0a8e851b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:01.723517 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4b79d016-c925-4ccf-adc9-c8582026edc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4f1ca4842b54591ae328adb0a8e851b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:01.733182 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4b79d016-c925-4ccf-adc9-c8582026edc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4f1ca4842b54591ae328adb0a8e851b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:01.734283 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4b79d016-c925-4ccf-adc9-c8582026edc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4f1ca4842b54591ae328adb0a8e851b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:01.742553 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-4b79d016-c925-4ccf-adc9-c8582026edc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4f1ca4842b54591ae328adb0a8e851b'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:01.765545 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1271/2539] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:44:01 2022] DELETE /identity/v3/users/c4f1ca4842b54591ae328adb0a8e851b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:44:01.770815 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220e798-95c7-4776-81bf-d030a85e81cf None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:01.771288 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220e798-95c7-4776-81bf-d030a85e81cf None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:01.771419 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220e798-95c7-4776-81bf-d030a85e81cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:01.806396 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-b220e798-95c7-4776-81bf-d030a85e81cf None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:44:01.807787 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b220e798-95c7-4776-81bf-d030a85e81cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:01.815816 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:44:01.928920 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1269/2540] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:44:01.940681 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79d016-c925-4ccf-adc9-c8582026edc6 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:01.942137 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-4b79d016-c925-4ccf-adc9-c8582026edc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:02.005000 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:44:02.066770 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79d016-c925-4ccf-adc9-c8582026edc6 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:02.076943 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972da8d-adb9-4c86-93fe-e064bfa422fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:02.076943 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972da8d-adb9-4c86-93fe-e064bfa422fd service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:02.076943 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972da8d-adb9-4c86-93fe-e064bfa422fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:02.076943 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972da8d-adb9-4c86-93fe-e064bfa422fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:02.084286 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e972da8d-adb9-4c86-93fe-e064bfa422fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:02.086220 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e972da8d-adb9-4c86-93fe-e064bfa422fd service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:02.098181 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1272/2541] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:44:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:44:02.725946 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220e798-95c7-4776-81bf-d030a85e81cf None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:02.727318 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-b220e798-95c7-4776-81bf-d030a85e81cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:02.745441 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220e798-95c7-4776-81bf-d030a85e81cf None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:02.751044 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f055f3c-7f82-4928-ad04-67b2d2b9371a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:02.752259 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f055f3c-7f82-4928-ad04-67b2d2b9371a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:02.752598 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f055f3c-7f82-4928-ad04-67b2d2b9371a admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:02.752894 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f055f3c-7f82-4928-ad04-67b2d2b9371a admin admin] PATH_INFO: `/v3/projects/aad84947a40f4e7eada99082ee91ab34` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:02.754094 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f055f3c-7f82-4928-ad04-67b2d2b9371a admin admin] RBAC: Authorizing `identity:delete_project(project_id=aad84947a40f4e7eada99082ee91ab34)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:02.756037 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f055f3c-7f82-4928-ad04-67b2d2b9371a admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:02.812906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8f055f3c-7f82-4928-ad04-67b2d2b9371a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aad84947a40f4e7eada99082ee91ab34', 'request_id': 'req-8f055f3c-7f82-4928-ad04-67b2d2b9371a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:02.819700 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8f055f3c-7f82-4928-ad04-67b2d2b9371a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aad84947a40f4e7eada99082ee91ab34', 'request_id': 'req-8f055f3c-7f82-4928-ad04-67b2d2b9371a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:02.828095 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8f055f3c-7f82-4928-ad04-67b2d2b9371a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aad84947a40f4e7eada99082ee91ab34', 'request_id': 'req-8f055f3c-7f82-4928-ad04-67b2d2b9371a'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:02.843478 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8f055f3c-7f82-4928-ad04-67b2d2b9371a admin admin] The token cache is being invalidate because project aad84947a40f4e7eada99082ee91ab34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:44:02.844253 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8f055f3c-7f82-4928-ad04-67b2d2b9371a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:02.845154 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8f055f3c-7f82-4928-ad04-67b2d2b9371a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:02.849055 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1270/2542] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:44:02 2022] DELETE /identity/v3/projects/aad84947a40f4e7eada99082ee91ab34 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:44:02.851586 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972da8d-adb9-4c86-93fe-e064bfa422fd service neutron] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:02.863656 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e972da8d-adb9-4c86-93fe-e064bfa422fd service neutron] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:02.931076 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd991a-b6db-4477-9926-894633328ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:02.931829 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd991a-b6db-4477-9926-894633328ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:02.932181 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd991a-b6db-4477-9926-894633328ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:02.932530 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd991a-b6db-4477-9926-894633328ed3 admin admin] PATH_INFO: `/v3/users/030267a47ab24bd2ae40aa6d2954f4c8` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:02.933949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd991a-b6db-4477-9926-894633328ed3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=030267a47ab24bd2ae40aa6d2954f4c8)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:02.935968 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd991a-b6db-4477-9926-894633328ed3 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:02.993317 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-3efd991a-b6db-4477-9926-894633328ed3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '030267a47ab24bd2ae40aa6d2954f4c8'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:02.994443 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-3efd991a-b6db-4477-9926-894633328ed3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '030267a47ab24bd2ae40aa6d2954f4c8'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:03.007545 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-3efd991a-b6db-4477-9926-894633328ed3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '030267a47ab24bd2ae40aa6d2954f4c8'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:03.008811 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-3efd991a-b6db-4477-9926-894633328ed3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '030267a47ab24bd2ae40aa6d2954f4c8'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:03.017577 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-3efd991a-b6db-4477-9926-894633328ed3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '030267a47ab24bd2ae40aa6d2954f4c8'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:03.041992 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1273/2543] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:44:02 2022] DELETE /identity/v3/users/030267a47ab24bd2ae40aa6d2954f4c8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:44:03.325762 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b6e23-343e-4e76-8dc5-faf21da53b2b None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:03.326057 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b6e23-343e-4e76-8dc5-faf21da53b2b None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:03.326354 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b6e23-343e-4e76-8dc5-faf21da53b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:03.356316 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-821b6e23-343e-4e76-8dc5-faf21da53b2b None None] MFA Rules not processed for user `7f7a6b4a92ff4232843913a1dbd7f1c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:44:03.357677 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-821b6e23-343e-4e76-8dc5-faf21da53b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:03.388121 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd991a-b6db-4477-9926-894633328ed3 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:03.389893 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-3efd991a-b6db-4477-9926-894633328ed3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:03.435116 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1271/2544] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:44:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:44:03.439694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b6e23-343e-4e76-8dc5-faf21da53b2b None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:03.441392 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-821b6e23-343e-4e76-8dc5-faf21da53b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:03.462062 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:44:03.474111 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:44:03.485150 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd991a-b6db-4477-9926-894633328ed3 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:03.490071 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd76c3-d857-44c9-bf7f-d4ddb40a02a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:03.491187 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd76c3-d857-44c9-bf7f-d4ddb40a02a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:03.491470 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd76c3-d857-44c9-bf7f-d4ddb40a02a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:03.491766 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd76c3-d857-44c9-bf7f-d4ddb40a02a8 admin admin] PATH_INFO: `/v3/projects/9fcc02cc00764387bc20193afd67492c` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:03.492911 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd76c3-d857-44c9-bf7f-d4ddb40a02a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fcc02cc00764387bc20193afd67492c)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:03.494843 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd76c3-d857-44c9-bf7f-d4ddb40a02a8 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:03.524400 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b6e23-343e-4e76-8dc5-faf21da53b2b None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:03.529761 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6da7c-f6a1-4805-a42c-e87da05b57b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:03.530472 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6da7c-f6a1-4805-a42c-e87da05b57b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:03.530753 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6da7c-f6a1-4805-a42c-e87da05b57b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:03.531025 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6da7c-f6a1-4805-a42c-e87da05b57b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:03.531821 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6da7c-f6a1-4805-a42c-e87da05b57b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:03.533814 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6da7c-f6a1-4805-a42c-e87da05b57b2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:03.561541 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-37dd76c3-d857-44c9-bf7f-d4ddb40a02a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fcc02cc00764387bc20193afd67492c', 'request_id': 'req-37dd76c3-d857-44c9-bf7f-d4ddb40a02a8'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:03.565064 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1272/2545] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:44:03 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:44:03.568576 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-37dd76c3-d857-44c9-bf7f-d4ddb40a02a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fcc02cc00764387bc20193afd67492c', 'request_id': 'req-37dd76c3-d857-44c9-bf7f-d4ddb40a02a8'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:03.569019 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6da7c-f6a1-4805-a42c-e87da05b57b2 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:03.577593 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-37dd76c3-d857-44c9-bf7f-d4ddb40a02a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fcc02cc00764387bc20193afd67492c', 'request_id': 'req-37dd76c3-d857-44c9-bf7f-d4ddb40a02a8'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:03.590421 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-37dd76c3-d857-44c9-bf7f-d4ddb40a02a8 admin admin] The token cache is being invalidate because project 9fcc02cc00764387bc20193afd67492c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:44:03.590714 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-37dd76c3-d857-44c9-bf7f-d4ddb40a02a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:03.591751 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-37dd76c3-d857-44c9-bf7f-d4ddb40a02a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:03.596764 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1274/2546] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:44:03 2022] DELETE /identity/v3/projects/9fcc02cc00764387bc20193afd67492c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:44:03.598382 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd76c3-d857-44c9-bf7f-d4ddb40a02a8 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:03.600355 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-37dd76c3-d857-44c9-bf7f-d4ddb40a02a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:03.636497 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6da7c-f6a1-4805-a42c-e87da05b57b2 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:03.641829 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1088c7f-97ea-4075-b35c-96fecc901b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:03.642521 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1088c7f-97ea-4075-b35c-96fecc901b6d admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:03.642804 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1088c7f-97ea-4075-b35c-96fecc901b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:03.643083 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1088c7f-97ea-4075-b35c-96fecc901b6d admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:03.643726 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1088c7f-97ea-4075-b35c-96fecc901b6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:03.645689 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1088c7f-97ea-4075-b35c-96fecc901b6d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:03.670117 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1273/2547] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:44:03 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 20 13:44:03.674401 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1088c7f-97ea-4075-b35c-96fecc901b6d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:03.676142 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c1088c7f-97ea-4075-b35c-96fecc901b6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:03.677442 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd76c3-d857-44c9-bf7f-d4ddb40a02a8 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:03.682712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7255a-a040-4e47-981a-69a17f521b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:03.683188 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7255a-a040-4e47-981a-69a17f521b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:03.683582 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7255a-a040-4e47-981a-69a17f521b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:03.684054 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7255a-a040-4e47-981a-69a17f521b12 admin admin] PATH_INFO: `/v3/users/934a4e45bb9b47158ddb2546c7df57de` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:03.685233 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f7255a-a040-4e47-981a-69a17f521b12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=934a4e45bb9b47158ddb2546c7df57de)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:03.687396 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f7255a-a040-4e47-981a-69a17f521b12 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:03.690932 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1088c7f-97ea-4075-b35c-96fecc901b6d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:03.698182 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d1db6-039d-4c74-a53f-e7d006a3307d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:03.699076 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d1db6-039d-4c74-a53f-e7d006a3307d admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:03.699453 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d1db6-039d-4c74-a53f-e7d006a3307d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:03.699822 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d1db6-039d-4c74-a53f-e7d006a3307d admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:03.700636 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d1db6-039d-4c74-a53f-e7d006a3307d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:03.702451 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d1db6-039d-4c74-a53f-e7d006a3307d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:03.715950 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1274/2548] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:44:03.721167 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d1db6-039d-4c74-a53f-e7d006a3307d admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:03.733444 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d1db6-039d-4c74-a53f-e7d006a3307d admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:03.738877 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4546b-18f1-4851-b851-879e53eb0a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:03.741514 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4546b-18f1-4851-b851-879e53eb0a07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:03.742021 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4546b-18f1-4851-b851-879e53eb0a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:03.742506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4546b-18f1-4851-b851-879e53eb0a07 admin admin] PATH_INFO: `/v3/projects/e461387a7dbe4f3fabaab7634fe5f7c6/users/e6a9be479a28499fbd2d57e46d6c643d/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:03.758964 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-23f7255a-a040-4e47-981a-69a17f521b12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '934a4e45bb9b47158ddb2546c7df57de'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:03.759695 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-23f7255a-a040-4e47-981a-69a17f521b12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '934a4e45bb9b47158ddb2546c7df57de'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:03.768437 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-23f7255a-a040-4e47-981a-69a17f521b12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '934a4e45bb9b47158ddb2546c7df57de'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:03.769246 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-23f7255a-a040-4e47-981a-69a17f521b12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '934a4e45bb9b47158ddb2546c7df57de'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:03.770594 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4546b-18f1-4851-b851-879e53eb0a07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e461387a7dbe4f3fabaab7634fe5f7c6, user_id=e6a9be479a28499fbd2d57e46d6c643d, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:03.774616 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4546b-18f1-4851-b851-879e53eb0a07 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:03.776594 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-23f7255a-a040-4e47-981a-69a17f521b12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '934a4e45bb9b47158ddb2546c7df57de'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:03.789372 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:44:03.796600 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1275/2549] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:44:03 2022] PUT /identity/v3/projects/e461387a7dbe4f3fabaab7634fe5f7c6/users/e6a9be479a28499fbd2d57e46d6c643d/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:44:03.800177 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1275/2550] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:44:03 2022] DELETE /identity/v3/users/934a4e45bb9b47158ddb2546c7df57de => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:44:03.800650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4546b-18f1-4851-b851-879e53eb0a07 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:03.802421 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e5d4546b-18f1-4851-b851-879e53eb0a07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:03.881992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4546b-18f1-4851-b851-879e53eb0a07 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:03.914864 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:44:03.977613 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75611901-2956-4edc-bcd1-ac1fc7063eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:03.978245 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75611901-2956-4edc-bcd1-ac1fc7063eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:03.978535 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75611901-2956-4edc-bcd1-ac1fc7063eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:03.979021 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75611901-2956-4edc-bcd1-ac1fc7063eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:03.979868 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75611901-2956-4edc-bcd1-ac1fc7063eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:03.983296 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75611901-2956-4edc-bcd1-ac1fc7063eb2 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:04.002656 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1276/2551] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:44:04.007400 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7255a-a040-4e47-981a-69a17f521b12 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:04.023417 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7255a-a040-4e47-981a-69a17f521b12 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:04.028934 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95099f-1538-47f1-9b30-3e1db367e70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:04.029621 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95099f-1538-47f1-9b30-3e1db367e70b admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:04.029902 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95099f-1538-47f1-9b30-3e1db367e70b admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:04.030331 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95099f-1538-47f1-9b30-3e1db367e70b admin admin] PATH_INFO: `/v3/projects/e461387a7dbe4f3fabaab7634fe5f7c6/users/e6a9be479a28499fbd2d57e46d6c643d/roles/2bcf35e1bc5f4634b0aa6411fad2f43f` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:04.033367 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b95099f-1538-47f1-9b30-3e1db367e70b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e461387a7dbe4f3fabaab7634fe5f7c6, user_id=e6a9be479a28499fbd2d57e46d6c643d, role_id=2bcf35e1bc5f4634b0aa6411fad2f43f)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:04.036248 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b95099f-1538-47f1-9b30-3e1db367e70b admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:04.049993 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1276/2552] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:44:04 2022] PUT /identity/v3/projects/e461387a7dbe4f3fabaab7634fe5f7c6/users/e6a9be479a28499fbd2d57e46d6c643d/roles/2bcf35e1bc5f4634b0aa6411fad2f43f => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 20 13:44:04.149614 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75611901-2956-4edc-bcd1-ac1fc7063eb2 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:04.151558 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-75611901-2956-4edc-bcd1-ac1fc7063eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:04.180992 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75611901-2956-4edc-bcd1-ac1fc7063eb2 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:04.266779 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa3f37-6cd3-46f1-82da-dc522a681ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:04.268071 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa3f37-6cd3-46f1-82da-dc522a681ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:04.269134 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa3f37-6cd3-46f1-82da-dc522a681ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:04.269426 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa3f37-6cd3-46f1-82da-dc522a681ffe admin admin] PATH_INFO: `/v3/projects/4aca3222198e493aad90346ddceffef9` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:04.271336 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa3f37-6cd3-46f1-82da-dc522a681ffe admin admin] RBAC: Authorizing `identity:delete_project(project_id=4aca3222198e493aad90346ddceffef9)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:04.273533 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa3f37-6cd3-46f1-82da-dc522a681ffe admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:04.339995 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-42aa3f37-6cd3-46f1-82da-dc522a681ffe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4aca3222198e493aad90346ddceffef9', 'request_id': 'req-42aa3f37-6cd3-46f1-82da-dc522a681ffe'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:04.348204 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-42aa3f37-6cd3-46f1-82da-dc522a681ffe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4aca3222198e493aad90346ddceffef9', 'request_id': 'req-42aa3f37-6cd3-46f1-82da-dc522a681ffe'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:04.357261 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-42aa3f37-6cd3-46f1-82da-dc522a681ffe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4aca3222198e493aad90346ddceffef9', 'request_id': 'req-42aa3f37-6cd3-46f1-82da-dc522a681ffe'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:04.371669 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-42aa3f37-6cd3-46f1-82da-dc522a681ffe admin admin] The token cache is being invalidate because project 4aca3222198e493aad90346ddceffef9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:44:04.371966 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-42aa3f37-6cd3-46f1-82da-dc522a681ffe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:04.372644 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-42aa3f37-6cd3-46f1-82da-dc522a681ffe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:04.374548 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1277/2553] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:44:04 2022] DELETE /identity/v3/projects/4aca3222198e493aad90346ddceffef9 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:44:08.921193 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf32e72-db7a-49a7-beb4-2a803dcaa344 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:08.923290 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf32e72-db7a-49a7-beb4-2a803dcaa344 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:08.923801 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf32e72-db7a-49a7-beb4-2a803dcaa344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:08.975941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-9cf32e72-db7a-49a7-beb4-2a803dcaa344 None None] MFA Rules not processed for user `e6a9be479a28499fbd2d57e46d6c643d`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:44:08.977940 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-9cf32e72-db7a-49a7-beb4-2a803dcaa344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:09.053050 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1277/2554] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:44:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:44:09.136898 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa3f37-6cd3-46f1-82da-dc522a681ffe admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:09.138506 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-42aa3f37-6cd3-46f1-82da-dc522a681ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:09.185073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:44:09.255234 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa3f37-6cd3-46f1-82da-dc522a681ffe admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:09.260741 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06eb31-211c-4e0a-8c34-c64efaddadca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:09.261415 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06eb31-211c-4e0a-8c34-c64efaddadca service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:09.261686 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06eb31-211c-4e0a-8c34-c64efaddadca service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:09.262041 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06eb31-211c-4e0a-8c34-c64efaddadca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:09.263649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-1c06eb31-211c-4e0a-8c34-c64efaddadca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:09.322364 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06eb31-211c-4e0a-8c34-c64efaddadca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:09.323935 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06eb31-211c-4e0a-8c34-c64efaddadca service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:09.333874 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1278/2555] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:44:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 20 13:44:10.458370 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ebacd-86dc-4e23-860a-94d8541e1994 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:10.459324 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ebacd-86dc-4e23-860a-94d8541e1994 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:10.459814 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ebacd-86dc-4e23-860a-94d8541e1994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:10.481649 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-668ebacd-86dc-4e23-860a-94d8541e1994 None None] MFA Rules not processed for user `7f7a6b4a92ff4232843913a1dbd7f1c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:44:10.483191 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-668ebacd-86dc-4e23-860a-94d8541e1994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:10.492077 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1278/2556] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:44:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:44:12.708585 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287db273-561a-4d64-989f-7f505ca5b29a None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:12.709068 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287db273-561a-4d64-989f-7f505ca5b29a None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:12.709200 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287db273-561a-4d64-989f-7f505ca5b29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:12.739290 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-287db273-561a-4d64-989f-7f505ca5b29a None None] MFA Rules not processed for user `85bb137d0fb74031a49a5c629b32c32b`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:44:12.740750 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-287db273-561a-4d64-989f-7f505ca5b29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:12.799956 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1279/2557] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 20 13:44:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 20 13:44:12.821044 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ebacd-86dc-4e23-860a-94d8541e1994 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:12.822456 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-668ebacd-86dc-4e23-860a-94d8541e1994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:12.896646 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ebacd-86dc-4e23-860a-94d8541e1994 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:12.901767 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061df9b4-6099-4f2f-9566-15f90f2ba343 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9834211488354651803000c7e8fab80b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:12.902591 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061df9b4-6099-4f2f-9566-15f90f2ba343 service swift] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:12.902876 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061df9b4-6099-4f2f-9566-15f90f2ba343 service swift] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:12.903154 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061df9b4-6099-4f2f-9566-15f90f2ba343 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:12.912991 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061df9b4-6099-4f2f-9566-15f90f2ba343 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:12.914605 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061df9b4-6099-4f2f-9566-15f90f2ba343 service swift] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:12.924801 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1279/2558] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 20 13:44:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 13:44:13.460550 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58043b38-013b-4b51-a14f-a851bcd35bb3 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:13.460853 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58043b38-013b-4b51-a14f-a851bcd35bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:13.461137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58043b38-013b-4b51-a14f-a851bcd35bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:13.481393 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-58043b38-013b-4b51-a14f-a851bcd35bb3 None None] MFA Rules not processed for user `85bb137d0fb74031a49a5c629b32c32b`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:44:13.482630 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-58043b38-013b-4b51-a14f-a851bcd35bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:13.490799 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1280/2559] 10.222.0.23 () {58 vars in 967 bytes} [Thu Oct 20 13:44:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:44:13.503533 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061df9b4-6099-4f2f-9566-15f90f2ba343 service swift] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:13.514604 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061df9b4-6099-4f2f-9566-15f90f2ba343 service swift] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:13.520220 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59f400-6798-4ab6-9545-7e65988144f2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9834211488354651803000c7e8fab80b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:13.521465 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59f400-6798-4ab6-9545-7e65988144f2 service swift] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:13.521746 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59f400-6798-4ab6-9545-7e65988144f2 service swift] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:13.522051 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59f400-6798-4ab6-9545-7e65988144f2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:13.531208 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59f400-6798-4ab6-9545-7e65988144f2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:13.532941 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d59f400-6798-4ab6-9545-7e65988144f2 service swift] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:13.541017 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1280/2560] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 20 13:44:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 13:44:13.779279 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9d8f8-e974-4b6a-b861-8cef479f0d0f None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:13.779765 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9d8f8-e974-4b6a-b861-8cef479f0d0f None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:13.779946 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9d8f8-e974-4b6a-b861-8cef479f0d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:13.800260 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-03b9d8f8-e974-4b6a-b861-8cef479f0d0f None None] MFA Rules not processed for user `85bb137d0fb74031a49a5c629b32c32b`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:44:13.801493 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-03b9d8f8-e974-4b6a-b861-8cef479f0d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:13.808598 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1281/2561] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 20 13:44:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:44:13.821242 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59f400-6798-4ab6-9545-7e65988144f2 service swift] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:13.834456 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d59f400-6798-4ab6-9545-7e65988144f2 service swift] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:13.840466 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e54ad-10b1-4d60-8810-9b2e8d0811da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9834211488354651803000c7e8fab80b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:13.841120 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e54ad-10b1-4d60-8810-9b2e8d0811da service swift] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:13.841403 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e54ad-10b1-4d60-8810-9b2e8d0811da service swift] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:13.841795 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e54ad-10b1-4d60-8810-9b2e8d0811da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:13.851455 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02e54ad-10b1-4d60-8810-9b2e8d0811da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:13.853101 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02e54ad-10b1-4d60-8810-9b2e8d0811da service swift] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:13.861535 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1281/2562] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 20 13:44:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 13:44:14.193123 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32629a-6256-4698-91a8-981319fd4655 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:14.193441 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32629a-6256-4698-91a8-981319fd4655 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:14.193721 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32629a-6256-4698-91a8-981319fd4655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:14.216941 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-ae32629a-6256-4698-91a8-981319fd4655 None None] MFA Rules not processed for user `85bb137d0fb74031a49a5c629b32c32b`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:44:14.216941 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ae32629a-6256-4698-91a8-981319fd4655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:14.223757 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1282/2563] 10.222.0.23 () {58 vars in 967 bytes} [Thu Oct 20 13:44:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:44:14.236234 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e54ad-10b1-4d60-8810-9b2e8d0811da service swift] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:14.247338 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e54ad-10b1-4d60-8810-9b2e8d0811da service swift] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:14.253166 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da89baf-6057-47e7-a276-16650efa77c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9834211488354651803000c7e8fab80b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:14.253767 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da89baf-6057-47e7-a276-16650efa77c7 service swift] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:14.254048 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da89baf-6057-47e7-a276-16650efa77c7 service swift] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:14.254321 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da89baf-6057-47e7-a276-16650efa77c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:14.262625 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da89baf-6057-47e7-a276-16650efa77c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:14.264468 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da89baf-6057-47e7-a276-16650efa77c7 service swift] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:14.276183 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1282/2564] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 20 13:44:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 20 13:44:16.396708 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32629a-6256-4698-91a8-981319fd4655 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:16.398124 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-ae32629a-6256-4698-91a8-981319fd4655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:16.426152 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=100 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:44:16.464410 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32629a-6256-4698-91a8-981319fd4655 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:16.469499 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e88a57-c4a6-4ac5-8da5-586345389e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:16.470124 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e88a57-c4a6-4ac5-8da5-586345389e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:16.470399 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e88a57-c4a6-4ac5-8da5-586345389e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:16.470734 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e88a57-c4a6-4ac5-8da5-586345389e50 admin admin] PATH_INFO: `/v3/users/69c1e55e00094866897bc509ae65c6e5` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:16.471843 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e88a57-c4a6-4ac5-8da5-586345389e50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69c1e55e00094866897bc509ae65c6e5)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:16.473771 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e88a57-c4a6-4ac5-8da5-586345389e50 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:16.529188 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-77e88a57-c4a6-4ac5-8da5-586345389e50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69c1e55e00094866897bc509ae65c6e5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:16.529836 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-77e88a57-c4a6-4ac5-8da5-586345389e50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69c1e55e00094866897bc509ae65c6e5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:16.536743 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-77e88a57-c4a6-4ac5-8da5-586345389e50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69c1e55e00094866897bc509ae65c6e5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:16.543867 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-77e88a57-c4a6-4ac5-8da5-586345389e50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69c1e55e00094866897bc509ae65c6e5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:16.544632 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-77e88a57-c4a6-4ac5-8da5-586345389e50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69c1e55e00094866897bc509ae65c6e5'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:16.563643 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1283/2565] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:44:16 2022] DELETE /identity/v3/users/69c1e55e00094866897bc509ae65c6e5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:44:16.568075 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb547a4d-9648-49ef-8cea-387428ccb343 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:16.568544 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb547a4d-9648-49ef-8cea-387428ccb343 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:16.568644 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb547a4d-9648-49ef-8cea-387428ccb343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:16.587610 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-bb547a4d-9648-49ef-8cea-387428ccb343 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:44:16.588796 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-bb547a4d-9648-49ef-8cea-387428ccb343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:16.594297 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:44:16.651241 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1283/2566] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:44:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:44:16.661357 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e88a57-c4a6-4ac5-8da5-586345389e50 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:16.662803 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-77e88a57-c4a6-4ac5-8da5-586345389e50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:16.724459 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e88a57-c4a6-4ac5-8da5-586345389e50 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:16.728797 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c3702-f41b-4c7c-9115-93d3827a9512 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:16.729463 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c3702-f41b-4c7c-9115-93d3827a9512 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:16.729744 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c3702-f41b-4c7c-9115-93d3827a9512 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:16.730018 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c3702-f41b-4c7c-9115-93d3827a9512 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:16.740301 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c3702-f41b-4c7c-9115-93d3827a9512 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:16.741752 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c3702-f41b-4c7c-9115-93d3827a9512 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:16.752039 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1284/2567] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:44:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:44:16.922198 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb547a4d-9648-49ef-8cea-387428ccb343 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:16.926748 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-bb547a4d-9648-49ef-8cea-387428ccb343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:16.939234 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb547a4d-9648-49ef-8cea-387428ccb343 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:16.943778 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b371bc8-1505-4c28-954d-b0f20e5f48cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:16.944411 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b371bc8-1505-4c28-954d-b0f20e5f48cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:16.944749 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b371bc8-1505-4c28-954d-b0f20e5f48cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:16.945151 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b371bc8-1505-4c28-954d-b0f20e5f48cd admin admin] PATH_INFO: `/v3/projects/29dff2585070499ea1cdaa39016417d0` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:16.946306 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b371bc8-1505-4c28-954d-b0f20e5f48cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=29dff2585070499ea1cdaa39016417d0)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:16.948261 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b371bc8-1505-4c28-954d-b0f20e5f48cd admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:16.996588 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0b371bc8-1505-4c28-954d-b0f20e5f48cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29dff2585070499ea1cdaa39016417d0', 'request_id': 'req-0b371bc8-1505-4c28-954d-b0f20e5f48cd'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:17.003477 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0b371bc8-1505-4c28-954d-b0f20e5f48cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29dff2585070499ea1cdaa39016417d0', 'request_id': 'req-0b371bc8-1505-4c28-954d-b0f20e5f48cd'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:17.010222 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0b371bc8-1505-4c28-954d-b0f20e5f48cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29dff2585070499ea1cdaa39016417d0', 'request_id': 'req-0b371bc8-1505-4c28-954d-b0f20e5f48cd'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:17.020963 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0b371bc8-1505-4c28-954d-b0f20e5f48cd admin admin] The token cache is being invalidate because project 29dff2585070499ea1cdaa39016417d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:44:17.021284 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0b371bc8-1505-4c28-954d-b0f20e5f48cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:17.021924 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0b371bc8-1505-4c28-954d-b0f20e5f48cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:17.023410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1284/2568] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:44:16 2022] DELETE /identity/v3/projects/29dff2585070499ea1cdaa39016417d0 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:44:17.028515 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d1740-7a93-44b0-8f33-94926919f9b5 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:17.028801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d1740-7a93-44b0-8f33-94926919f9b5 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:17.029130 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d1740-7a93-44b0-8f33-94926919f9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:17.049055 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-c58d1740-7a93-44b0-8f33-94926919f9b5 None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:44:17.050286 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-c58d1740-7a93-44b0-8f33-94926919f9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:17.095550 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:44:17.115340 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1285/2569] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:44:17.122678 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846723f-4fc8-4b7f-85c0-75acaa31c71f None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:17.122969 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846723f-4fc8-4b7f-85c0-75acaa31c71f None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:17.123277 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846723f-4fc8-4b7f-85c0-75acaa31c71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:17.145434 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-7846723f-4fc8-4b7f-85c0-75acaa31c71f None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:44:17.146635 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-7846723f-4fc8-4b7f-85c0-75acaa31c71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:17.154445 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1285/2570] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:44:17.159801 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d1740-7a93-44b0-8f33-94926919f9b5 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:17.169319 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d1740-7a93-44b0-8f33-94926919f9b5 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:17.174386 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c41141-3bb6-4064-a748-968ee20f6eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:17.175007 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c41141-3bb6-4064-a748-968ee20f6eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:17.175288 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c41141-3bb6-4064-a748-968ee20f6eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:17.175561 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c41141-3bb6-4064-a748-968ee20f6eb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:17.176215 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c41141-3bb6-4064-a748-968ee20f6eb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81475) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 20 13:44:17.176519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c41141-3bb6-4064-a748-968ee20f6eb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:17.178156 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c41141-3bb6-4064-a748-968ee20f6eb9 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:17.190172 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1286/2571] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 20 13:44:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 20 13:44:17.194687 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846723f-4fc8-4b7f-85c0-75acaa31c71f None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:17.205658 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846723f-4fc8-4b7f-85c0-75acaa31c71f None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:17.210218 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b8e25-cb0f-4b45-988b-90fb2e9374af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:17.210811 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b8e25-cb0f-4b45-988b-90fb2e9374af admin admin] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:17.211086 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b8e25-cb0f-4b45-988b-90fb2e9374af admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:17.211356 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b8e25-cb0f-4b45-988b-90fb2e9374af admin admin] PATH_INFO: `/v3/projects` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:17.212041 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b8e25-cb0f-4b45-988b-90fb2e9374af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:17.213954 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b8e25-cb0f-4b45-988b-90fb2e9374af admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:17.242964 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1286/2572] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 20 13:44:17 2022] POST /identity/v3/projects => generated 389 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:44:17.247257 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c41141-3bb6-4064-a748-968ee20f6eb9 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:17.258163 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c41141-3bb6-4064-a748-968ee20f6eb9 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:17.318906 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad057439-2b25-4a84-876f-c3b100258d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:17.319517 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad057439-2b25-4a84-876f-c3b100258d40 admin admin] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:17.319799 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad057439-2b25-4a84-876f-c3b100258d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:17.320069 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad057439-2b25-4a84-876f-c3b100258d40 admin admin] PATH_INFO: `/v3/users` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:17.320704 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad057439-2b25-4a84-876f-c3b100258d40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:17.322851 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad057439-2b25-4a84-876f-c3b100258d40 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:17.348416 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1287/2573] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 20 13:44:17 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 20 13:44:17.352979 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b8e25-cb0f-4b45-988b-90fb2e9374af admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:17.363155 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b8e25-cb0f-4b45-988b-90fb2e9374af admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:17.368077 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135cf765-3992-4e9d-8f1f-f31b28e916b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:17.368633 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cf765-3992-4e9d-8f1f-f31b28e916b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:17.368949 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cf765-3992-4e9d-8f1f-f31b28e916b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:17.369228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cf765-3992-4e9d-8f1f-f31b28e916b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:17.369872 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135cf765-3992-4e9d-8f1f-f31b28e916b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:17.371639 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135cf765-3992-4e9d-8f1f-f31b28e916b7 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:17.383832 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1287/2574] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 20 13:44:17.387744 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad057439-2b25-4a84-876f-c3b100258d40 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:17.400392 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad057439-2b25-4a84-876f-c3b100258d40 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:17.405382 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91047645-eb10-432d-a93b-82f67132a313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:17.406206 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91047645-eb10-432d-a93b-82f67132a313 admin admin] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:17.406793 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91047645-eb10-432d-a93b-82f67132a313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:17.407071 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91047645-eb10-432d-a93b-82f67132a313 admin admin] PATH_INFO: `/v3/roles` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:17.407725 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91047645-eb10-432d-a93b-82f67132a313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:17.409378 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91047645-eb10-432d-a93b-82f67132a313 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:17.420847 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1288/2575] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 20 13:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 20 13:44:17.426746 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135cf765-3992-4e9d-8f1f-f31b28e916b7 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:17.437600 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135cf765-3992-4e9d-8f1f-f31b28e916b7 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:17.442573 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e71973-1b2f-48bb-a8dd-8b9cad1de500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:17.443212 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e71973-1b2f-48bb-a8dd-8b9cad1de500 admin admin] REQUEST_METHOD: `PUT` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:17.443497 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e71973-1b2f-48bb-a8dd-8b9cad1de500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:17.443776 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e71973-1b2f-48bb-a8dd-8b9cad1de500 admin admin] PATH_INFO: `/v3/projects/6c47b9997280426082bbcff1c27a5eea/users/f220d4887684414bb2e7d9abf25226d0/roles/5c47f5b95d6f47a78adf8f8e91562e4b` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:17.459175 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e71973-1b2f-48bb-a8dd-8b9cad1de500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c47b9997280426082bbcff1c27a5eea, user_id=f220d4887684414bb2e7d9abf25226d0, role_id=5c47f5b95d6f47a78adf8f8e91562e4b)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:17.462049 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e71973-1b2f-48bb-a8dd-8b9cad1de500 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:17.475920 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1288/2576] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 20 13:44:17 2022] PUT /identity/v3/projects/6c47b9997280426082bbcff1c27a5eea/users/f220d4887684414bb2e7d9abf25226d0/roles/5c47f5b95d6f47a78adf8f8e91562e4b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:44:17.486165 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d9831-138a-42b8-bce9-9f69524b425e None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:17.486526 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d9831-138a-42b8-bce9-9f69524b425e None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:17.486875 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d9831-138a-42b8-bce9-9f69524b425e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:17.517970 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-907d9831-138a-42b8-bce9-9f69524b425e None None] MFA Rules not processed for user `f220d4887684414bb2e7d9abf25226d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:44:17.519168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-907d9831-138a-42b8-bce9-9f69524b425e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:17.584822 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1289/2577] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:44:17 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:44:17.595539 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e71973-1b2f-48bb-a8dd-8b9cad1de500 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:17.597220 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-d6e71973-1b2f-48bb-a8dd-8b9cad1de500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:17.632853 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:44:17.657436 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e71973-1b2f-48bb-a8dd-8b9cad1de500 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:17.661068 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5111004-87d9-4090-84ad-d5beef429450 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349ba417c3fc49ecaa11a65066dc54f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:17.661672 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5111004-87d9-4090-84ad-d5beef429450 service cinder] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:17.661947 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5111004-87d9-4090-84ad-d5beef429450 service cinder] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:17.662218 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5111004-87d9-4090-84ad-d5beef429450 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:17.670788 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5111004-87d9-4090-84ad-d5beef429450 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:17.673021 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5111004-87d9-4090-84ad-d5beef429450 service cinder] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:17.681606 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1289/2578] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:44:17 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:44:27.581901 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:44:27.677966 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=12 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:44:37.253098 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d9831-138a-42b8-bce9-9f69524b425e None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:37.315183 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d9831-138a-42b8-bce9-9f69524b425e None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:37.319680 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb2f3f6-16f5-4281-a1bb-a4ab0cdc7dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:37.320302 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2f3f6-16f5-4281-a1bb-a4ab0cdc7dc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:37.320575 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2f3f6-16f5-4281-a1bb-a4ab0cdc7dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:37.320850 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2f3f6-16f5-4281-a1bb-a4ab0cdc7dc5 admin admin] PATH_INFO: `/v3/users/f220d4887684414bb2e7d9abf25226d0` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:37.321983 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb2f3f6-16f5-4281-a1bb-a4ab0cdc7dc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f220d4887684414bb2e7d9abf25226d0)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:37.323876 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb2f3f6-16f5-4281-a1bb-a4ab0cdc7dc5 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:37.369580 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-afb2f3f6-16f5-4281-a1bb-a4ab0cdc7dc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f220d4887684414bb2e7d9abf25226d0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:37.370236 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-afb2f3f6-16f5-4281-a1bb-a4ab0cdc7dc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f220d4887684414bb2e7d9abf25226d0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:37.377654 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-afb2f3f6-16f5-4281-a1bb-a4ab0cdc7dc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f220d4887684414bb2e7d9abf25226d0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:37.385162 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-afb2f3f6-16f5-4281-a1bb-a4ab0cdc7dc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f220d4887684414bb2e7d9abf25226d0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:37.385772 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-afb2f3f6-16f5-4281-a1bb-a4ab0cdc7dc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f220d4887684414bb2e7d9abf25226d0'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:37.403412 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1290/2579] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:44:37 2022] DELETE /identity/v3/users/f220d4887684414bb2e7d9abf25226d0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:44:37.408605 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d9467-48ec-43e2-954c-899ad7639bdf None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:37.409125 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d9467-48ec-43e2-954c-899ad7639bdf None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:37.409242 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d9467-48ec-43e2-954c-899ad7639bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:37.429511 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-d51d9467-48ec-43e2-954c-899ad7639bdf None None] MFA Rules not processed for user `b0595b7096954e6a802224bb45e3f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:44:37.430711 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-d51d9467-48ec-43e2-954c-899ad7639bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:37.488881 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1290/2580] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 20 13:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:44:37.501221 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb2f3f6-16f5-4281-a1bb-a4ab0cdc7dc5 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:37.502581 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-afb2f3f6-16f5-4281-a1bb-a4ab0cdc7dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:37.559064 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb2f3f6-16f5-4281-a1bb-a4ab0cdc7dc5 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:37.563720 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c90af-f8c3-4f75-b496-29301524834f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:37.564318 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c90af-f8c3-4f75-b496-29301524834f service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:37.564597 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c90af-f8c3-4f75-b496-29301524834f service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:37.564886 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c90af-f8c3-4f75-b496-29301524834f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:37.573763 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c90af-f8c3-4f75-b496-29301524834f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:37.575185 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997c90af-f8c3-4f75-b496-29301524834f service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:37.585276 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1291/2581] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:44:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:44:37.849951 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d9467-48ec-43e2-954c-899ad7639bdf None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:37.851666 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-d51d9467-48ec-43e2-954c-899ad7639bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:37.867030 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d9467-48ec-43e2-954c-899ad7639bdf None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:37.876980 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534c401-1a46-4902-9afc-df8017739ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:37.877637 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534c401-1a46-4902-9afc-df8017739ad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:37.877914 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534c401-1a46-4902-9afc-df8017739ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:37.878184 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534c401-1a46-4902-9afc-df8017739ad8 admin admin] PATH_INFO: `/v3/projects/6c47b9997280426082bbcff1c27a5eea` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:37.879317 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d534c401-1a46-4902-9afc-df8017739ad8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c47b9997280426082bbcff1c27a5eea)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:37.881268 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d534c401-1a46-4902-9afc-df8017739ad8 admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:37.931097 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-d534c401-1a46-4902-9afc-df8017739ad8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c47b9997280426082bbcff1c27a5eea', 'request_id': 'req-d534c401-1a46-4902-9afc-df8017739ad8'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:37.938046 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-d534c401-1a46-4902-9afc-df8017739ad8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c47b9997280426082bbcff1c27a5eea', 'request_id': 'req-d534c401-1a46-4902-9afc-df8017739ad8'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:37.945371 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-d534c401-1a46-4902-9afc-df8017739ad8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c47b9997280426082bbcff1c27a5eea', 'request_id': 'req-d534c401-1a46-4902-9afc-df8017739ad8'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:37.955779 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-d534c401-1a46-4902-9afc-df8017739ad8 admin admin] The token cache is being invalidate because project 6c47b9997280426082bbcff1c27a5eea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81474) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:44:37.956064 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-d534c401-1a46-4902-9afc-df8017739ad8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:37.956684 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-d534c401-1a46-4902-9afc-df8017739ad8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:44:37.958193 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1291/2582] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:44:37 2022] DELETE /identity/v3/projects/6c47b9997280426082bbcff1c27a5eea => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:44:47.581021 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:44:47.952536 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:44:48.629823 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c90af-f8c3-4f75-b496-29301524834f service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:44:48.638604 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c90af-f8c3-4f75-b496-29301524834f service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:44:48.686432 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6259713-dd47-4af6-89d9-56685eb825b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:44:48.687029 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6259713-dd47-4af6-89d9-56685eb825b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:44:48.687310 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6259713-dd47-4af6-89d9-56685eb825b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:44:48.687584 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6259713-dd47-4af6-89d9-56685eb825b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:44:48.689336 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-e6259713-dd47-4af6-89d9-56685eb825b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:44:48.750518 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6259713-dd47-4af6-89d9-56685eb825b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:44:48.751940 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6259713-dd47-4af6-89d9-56685eb825b9 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:44:48.761204 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1292/2583] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:44:48 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:44:58.757014 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=50 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:46:43.872291 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534c401-1a46-4902-9afc-df8017739ad8 admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:46:43.873723 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-d534c401-1a46-4902-9afc-df8017739ad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:46:43.886405 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534c401-1a46-4902-9afc-df8017739ad8 admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:46:43.889924 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81946900-4e37-477e-9290-6cdf7884e29d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:46:43.890509 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81946900-4e37-477e-9290-6cdf7884e29d service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:46:43.890782 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81946900-4e37-477e-9290-6cdf7884e29d service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:46:43.891047 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81946900-4e37-477e-9290-6cdf7884e29d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:46:43.893399 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-81946900-4e37-477e-9290-6cdf7884e29d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:46:43.902715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81946900-4e37-477e-9290-6cdf7884e29d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:46:43.904136 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81946900-4e37-477e-9290-6cdf7884e29d service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:46:43.912213 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1292/2584] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:46:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:46:48.092650 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6259713-dd47-4af6-89d9-56685eb825b9 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:46:48.101442 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6259713-dd47-4af6-89d9-56685eb825b9 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:46:48.105037 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984ffcf-1873-4b86-ba7b-6ed93d37d05e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:46:48.105626 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984ffcf-1873-4b86-ba7b-6ed93d37d05e service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:46:48.105902 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984ffcf-1873-4b86-ba7b-6ed93d37d05e service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:46:48.106168 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984ffcf-1873-4b86-ba7b-6ed93d37d05e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:46:48.107858 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f984ffcf-1873-4b86-ba7b-6ed93d37d05e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:46:48.176848 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984ffcf-1873-4b86-ba7b-6ed93d37d05e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:46:48.178194 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984ffcf-1873-4b86-ba7b-6ed93d37d05e service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:46:48.188952 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1293/2585] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:46:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:46:53.908410 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=6 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:46:58.184164 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=34 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:47:02.407301 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518249e6-b570-4cdb-b10c-38d6277a90fb None None] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:47:02.407889 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518249e6-b570-4cdb-b10c-38d6277a90fb None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:47:02.408014 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518249e6-b570-4cdb-b10c-38d6277a90fb None None] PATH_INFO: `/` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:47:02.409578 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1293/2586] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 20 13:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:47:02.415587 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281bccd-326a-40ce-a1af-05f78bd850d2 None None] REQUEST_METHOD: `POST` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:47:02.416071 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281bccd-326a-40ce-a1af-05f78bd850d2 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:47:02.416199 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281bccd-326a-40ce-a1af-05f78bd850d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:47:02.459692 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.auth.core [None req-f281bccd-326a-40ce-a1af-05f78bd850d2 None None] MFA Rules not processed for user `8a101adfc7a94bf1a4777350e9210a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81475) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:47:02.460945 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-f281bccd-326a-40ce-a1af-05f78bd850d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:47:02.467382 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1294/2587] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 20 13:47:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:47:02.481798 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518249e6-b570-4cdb-b10c-38d6277a90fb None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:47:02.493369 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518249e6-b570-4cdb-b10c-38d6277a90fb None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:47:02.497715 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6ee8b-cbc6-44c7-9e52-4143b2520db6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:47:02.498615 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6ee8b-cbc6-44c7-9e52-4143b2520db6 service neutron] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:47:02.498998 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6ee8b-cbc6-44c7-9e52-4143b2520db6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:47:02.499282 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6ee8b-cbc6-44c7-9e52-4143b2520db6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:47:02.509175 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6ee8b-cbc6-44c7-9e52-4143b2520db6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:47:02.511006 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6ee8b-cbc6-44c7-9e52-4143b2520db6 service neutron] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:47:02.519677 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1294/2588] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:47:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:47:02.608490 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225eeaa-e11c-4d86-9a25-7f070cf4f681 None None] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:47:02.608710 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225eeaa-e11c-4d86-9a25-7f070cf4f681 None None] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:47:02.609033 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225eeaa-e11c-4d86-9a25-7f070cf4f681 None None] PATH_INFO: `/` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:47:02.610373 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1295/2589] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 20 13:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 20 13:47:02.616363 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a19b00-e5f9-4c90-ad74-601ca37042e3 None None] REQUEST_METHOD: `POST` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:47:02.616648 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a19b00-e5f9-4c90-ad74-601ca37042e3 None None] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:47:02.616948 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a19b00-e5f9-4c90-ad74-601ca37042e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:47:02.655228 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.auth.core [None req-74a19b00-e5f9-4c90-ad74-601ca37042e3 None None] MFA Rules not processed for user `ee5c2a3855cb4a668177aa19f337778d`. Rule list: `[]` (Enabled: `True`). {{(pid=81474) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 20 13:47:02.656461 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-74a19b00-e5f9-4c90-ad74-601ca37042e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:47:02.665759 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1295/2590] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 20 13:47:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 20 13:47:02.677438 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225eeaa-e11c-4d86-9a25-7f070cf4f681 None None] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:47:02.688484 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225eeaa-e11c-4d86-9a25-7f070cf4f681 None None] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:47:02.692608 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ba13e-e4c1-4f5f-8689-3ef19b20e6c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a101adfc7a94bf1a4777350e9210a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:47:02.693236 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ba13e-e4c1-4f5f-8689-3ef19b20e6c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:47:02.693519 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ba13e-e4c1-4f5f-8689-3ef19b20e6c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:47:02.693796 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ba13e-e4c1-4f5f-8689-3ef19b20e6c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:47:02.703666 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0ba13e-e4c1-4f5f-8689-3ef19b20e6c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:47:02.705116 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0ba13e-e4c1-4f5f-8689-3ef19b20e6c8 service neutron] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:47:02.715806 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1296/2591] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:47:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:47:02.779910 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a19b00-e5f9-4c90-ad74-601ca37042e3 None None] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:47:02.781323 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-74a19b00-e5f9-4c90-ad74-601ca37042e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:47:02.853033 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a19b00-e5f9-4c90-ad74-601ca37042e3 None None] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:47:02.858038 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fab746-061d-40db-8bd0-ef37898fc0b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2440594934b64621a01c441133778155', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:47:02.858682 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fab746-061d-40db-8bd0-ef37898fc0b7 service placement] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:47:02.858909 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fab746-061d-40db-8bd0-ef37898fc0b7 service placement] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:47:02.859180 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fab746-061d-40db-8bd0-ef37898fc0b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:47:02.861175 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-56fab746-061d-40db-8bd0-ef37898fc0b7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:47:02.872850 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fab746-061d-40db-8bd0-ef37898fc0b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:47:02.874453 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fab746-061d-40db-8bd0-ef37898fc0b7 service placement] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:47:02.884692 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1296/2592] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:47:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:47:04.578694 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ba13e-e4c1-4f5f-8689-3ef19b20e6c8 service neutron] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:47:04.580222 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-bb0ba13e-e4c1-4f5f-8689-3ef19b20e6c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:47:04.599913 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ba13e-e4c1-4f5f-8689-3ef19b20e6c8 service neutron] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:47:04.605466 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254cfaff-e232-46d1-96a4-74b8630a8174 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:47:04.606073 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254cfaff-e232-46d1-96a4-74b8630a8174 service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:47:04.606344 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254cfaff-e232-46d1-96a4-74b8630a8174 service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:47:04.606613 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254cfaff-e232-46d1-96a4-74b8630a8174 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:47:04.608464 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-254cfaff-e232-46d1-96a4-74b8630a8174 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:47:04.624751 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254cfaff-e232-46d1-96a4-74b8630a8174 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:47:04.626369 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254cfaff-e232-46d1-96a4-74b8630a8174 service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:47:04.638944 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1297/2593] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:47:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:47:12.880457 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=47 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:47:14.634478 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=21 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:48:26.767688 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fab746-061d-40db-8bd0-ef37898fc0b7 service placement] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:48:26.781546 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fab746-061d-40db-8bd0-ef37898fc0b7 service placement] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:48:26.787043 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab0bdf-743b-40bb-a994-163fa950c18b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:48:26.788830 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab0bdf-743b-40bb-a994-163fa950c18b service nova] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:48:26.789139 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab0bdf-743b-40bb-a994-163fa950c18b service nova] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:48:26.789463 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab0bdf-743b-40bb-a994-163fa950c18b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:48:26.791163 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-deab0bdf-743b-40bb-a994-163fa950c18b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:48:26.804126 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab0bdf-743b-40bb-a994-163fa950c18b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:48:26.805870 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab0bdf-743b-40bb-a994-163fa950c18b service nova] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:48:26.817230 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1297/2594] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:48:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:48:29.145123 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254cfaff-e232-46d1-96a4-74b8630a8174 service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:48:29.156790 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254cfaff-e232-46d1-96a4-74b8630a8174 service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:48:29.163060 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3cca5-fa91-4642-9daa-5b693435876e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee5c2a3855cb4a668177aa19f337778d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:48:29.163730 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3cca5-fa91-4642-9daa-5b693435876e service nova] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:48:29.164034 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3cca5-fa91-4642-9daa-5b693435876e service nova] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:48:29.164279 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3cca5-fa91-4642-9daa-5b693435876e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:48:29.166141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-a7f3cca5-fa91-4642-9daa-5b693435876e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:48:29.180216 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f3cca5-fa91-4642-9daa-5b693435876e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:48:29.181702 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f3cca5-fa91-4642-9daa-5b693435876e service nova] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:48:29.192902 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1298/2595] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:48:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:48:33.909385 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab0bdf-743b-40bb-a994-163fa950c18b service nova] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:48:33.933859 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab0bdf-743b-40bb-a994-163fa950c18b service nova] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:48:33.939849 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d69b6-9e7f-4ddc-a158-ab2a4e3c666a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2440594934b64621a01c441133778155', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:48:33.945987 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d69b6-9e7f-4ddc-a158-ab2a4e3c666a service placement] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:48:33.946264 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d69b6-9e7f-4ddc-a158-ab2a4e3c666a service placement] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:48:33.946729 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d69b6-9e7f-4ddc-a158-ab2a4e3c666a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:48:33.948661 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-058d69b6-9e7f-4ddc-a158-ab2a4e3c666a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:48:33.961482 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058d69b6-9e7f-4ddc-a158-ab2a4e3c666a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:48:33.963626 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058d69b6-9e7f-4ddc-a158-ab2a4e3c666a service placement] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:48:33.986712 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1298/2596] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:48:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:48:36.776928 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3cca5-fa91-4642-9daa-5b693435876e service nova] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:48:36.791964 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3cca5-fa91-4642-9daa-5b693435876e service nova] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:48:36.813340 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906aab62-3008-469a-ad2f-9530e98b4eaa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2440594934b64621a01c441133778155', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:48:36.813649 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906aab62-3008-469a-ad2f-9530e98b4eaa service placement] REQUEST_METHOD: `GET` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:48:36.814141 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906aab62-3008-469a-ad2f-9530e98b4eaa service placement] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:48:36.814598 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906aab62-3008-469a-ad2f-9530e98b4eaa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:48:36.817128 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-906aab62-3008-469a-ad2f-9530e98b4eaa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:48:36.851068 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906aab62-3008-469a-ad2f-9530e98b4eaa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:48:36.852727 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906aab62-3008-469a-ad2f-9530e98b4eaa service placement] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:48:36.859582 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d69b6-9e7f-4ddc-a158-ab2a4e3c666a service placement] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:48:36.861868 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-058d69b6-9e7f-4ddc-a158-ab2a4e3c666a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:48:36.875592 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1299/2597] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:48:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:48:36.944124 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d69b6-9e7f-4ddc-a158-ab2a4e3c666a service placement] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:48:36.949071 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d2244-209f-4e56-81c5-8a1d45bc8a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:48:36.949707 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d2244-209f-4e56-81c5-8a1d45bc8a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:48:36.949992 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d2244-209f-4e56-81c5-8a1d45bc8a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:48:36.950269 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d2244-209f-4e56-81c5-8a1d45bc8a8c admin admin] PATH_INFO: `/v3/users/7f7a6b4a92ff4232843913a1dbd7f1c4` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:48:36.951399 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d2244-209f-4e56-81c5-8a1d45bc8a8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f7a6b4a92ff4232843913a1dbd7f1c4)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:48:36.953379 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d2244-209f-4e56-81c5-8a1d45bc8a8c admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:48:37.002233 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-672d2244-209f-4e56-81c5-8a1d45bc8a8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f7a6b4a92ff4232843913a1dbd7f1c4'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:48:37.002901 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-672d2244-209f-4e56-81c5-8a1d45bc8a8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f7a6b4a92ff4232843913a1dbd7f1c4'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:48:37.010616 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-672d2244-209f-4e56-81c5-8a1d45bc8a8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f7a6b4a92ff4232843913a1dbd7f1c4'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:48:37.011243 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-672d2244-209f-4e56-81c5-8a1d45bc8a8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f7a6b4a92ff4232843913a1dbd7f1c4'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:48:37.018469 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-672d2244-209f-4e56-81c5-8a1d45bc8a8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f7a6b4a92ff4232843913a1dbd7f1c4'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:48:37.036762 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1299/2598] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:48:36 2022] DELETE /identity/v3/users/7f7a6b4a92ff4232843913a1dbd7f1c4 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:48:37.212763 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906aab62-3008-469a-ad2f-9530e98b4eaa service placement] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:48:37.230762 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906aab62-3008-469a-ad2f-9530e98b4eaa service placement] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:48:37.286423 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444bcad-d053-463f-bb63-0018aa95bef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:48:37.287939 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444bcad-d053-463f-bb63-0018aa95bef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:48:37.287939 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444bcad-d053-463f-bb63-0018aa95bef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:48:37.288187 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444bcad-d053-463f-bb63-0018aa95bef7 admin admin] PATH_INFO: `/v3/projects/2de4676254c14b55aa2b7765255c69c9` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:48:37.289332 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444bcad-d053-463f-bb63-0018aa95bef7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2de4676254c14b55aa2b7765255c69c9)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:48:37.291241 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444bcad-d053-463f-bb63-0018aa95bef7 admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:48:37.341655 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8444bcad-d053-463f-bb63-0018aa95bef7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2de4676254c14b55aa2b7765255c69c9', 'request_id': 'req-8444bcad-d053-463f-bb63-0018aa95bef7'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:48:37.348169 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8444bcad-d053-463f-bb63-0018aa95bef7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2de4676254c14b55aa2b7765255c69c9', 'request_id': 'req-8444bcad-d053-463f-bb63-0018aa95bef7'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:48:37.355752 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8444bcad-d053-463f-bb63-0018aa95bef7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2de4676254c14b55aa2b7765255c69c9', 'request_id': 'req-8444bcad-d053-463f-bb63-0018aa95bef7'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:48:37.368059 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8444bcad-d053-463f-bb63-0018aa95bef7 admin admin] The token cache is being invalidate because project 2de4676254c14b55aa2b7765255c69c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:48:37.368315 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8444bcad-d053-463f-bb63-0018aa95bef7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:48:37.369005 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-8444bcad-d053-463f-bb63-0018aa95bef7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:48:37.370500 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1300/2599] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:48:37 2022] DELETE /identity/v3/projects/2de4676254c14b55aa2b7765255c69c9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:48:37.374462 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d2244-209f-4e56-81c5-8a1d45bc8a8c admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:48:37.375869 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-672d2244-209f-4e56-81c5-8a1d45bc8a8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:48:37.389781 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d2244-209f-4e56-81c5-8a1d45bc8a8c admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:48:37.437271 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:48:37.448223 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67248e-0619-4b19-b34b-79f4cddec11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:48:37.448849 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67248e-0619-4b19-b34b-79f4cddec11b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:48:37.449160 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67248e-0619-4b19-b34b-79f4cddec11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:48:37.449432 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67248e-0619-4b19-b34b-79f4cddec11b admin admin] PATH_INFO: `/v3/users/e6a9be479a28499fbd2d57e46d6c643d` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:48:37.450541 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d67248e-0619-4b19-b34b-79f4cddec11b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6a9be479a28499fbd2d57e46d6c643d)` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:48:37.452436 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d67248e-0619-4b19-b34b-79f4cddec11b admin admin] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:48:37.498862 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0d67248e-0619-4b19-b34b-79f4cddec11b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6a9be479a28499fbd2d57e46d6c643d'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:48:37.499505 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0d67248e-0619-4b19-b34b-79f4cddec11b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6a9be479a28499fbd2d57e46d6c643d'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:48:37.507614 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0d67248e-0619-4b19-b34b-79f4cddec11b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6a9be479a28499fbd2d57e46d6c643d'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:48:37.508274 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0d67248e-0619-4b19-b34b-79f4cddec11b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6a9be479a28499fbd2d57e46d6c643d'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:48:37.515995 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.notifications [None req-0d67248e-0619-4b19-b34b-79f4cddec11b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6a9be479a28499fbd2d57e46d6c643d'} {{(pid=81474) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:48:37.533014 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1300/2600] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 20 13:48:37 2022] DELETE /identity/v3/users/e6a9be479a28499fbd2d57e46d6c643d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:48:37.704236 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444bcad-d053-463f-bb63-0018aa95bef7 admin admin] Authenticating user token {{(pid=81475) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:48:37.705725 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.fernet_utils [None req-8444bcad-d053-463f-bb63-0018aa95bef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81475) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:48:37.718535 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444bcad-d053-463f-bb63-0018aa95bef7 admin admin] Validating token access rules against request {{(pid=81475) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:48:37.775783 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82909122-fe43-4334-b98f-6090e0edcb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0595b7096954e6a802224bb45e3f3cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7d49958a3a448d0bed98efad34bc0a2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81475) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:48:37.776421 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82909122-fe43-4334-b98f-6090e0edcb9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:48:37.776709 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82909122-fe43-4334-b98f-6090e0edcb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:48:37.777098 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82909122-fe43-4334-b98f-6090e0edcb9d admin admin] PATH_INFO: `/v3/projects/e461387a7dbe4f3fabaab7634fe5f7c6` {{(pid=81475) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:48:37.778220 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82909122-fe43-4334-b98f-6090e0edcb9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e461387a7dbe4f3fabaab7634fe5f7c6)` {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:48:37.780178 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82909122-fe43-4334-b98f-6090e0edcb9d admin admin] RBAC: Authorization granted {{(pid=81475) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:48:37.786477 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:48:37.829096 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-82909122-fe43-4334-b98f-6090e0edcb9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e461387a7dbe4f3fabaab7634fe5f7c6', 'request_id': 'req-82909122-fe43-4334-b98f-6090e0edcb9d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:48:37.836137 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-82909122-fe43-4334-b98f-6090e0edcb9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e461387a7dbe4f3fabaab7634fe5f7c6', 'request_id': 'req-82909122-fe43-4334-b98f-6090e0edcb9d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:48:37.843644 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-82909122-fe43-4334-b98f-6090e0edcb9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e461387a7dbe4f3fabaab7634fe5f7c6', 'request_id': 'req-82909122-fe43-4334-b98f-6090e0edcb9d'} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:48:37.854481 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-82909122-fe43-4334-b98f-6090e0edcb9d admin admin] The token cache is being invalidate because project e461387a7dbe4f3fabaab7634fe5f7c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81475) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 20 13:48:37.854796 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-82909122-fe43-4334-b98f-6090e0edcb9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:48:37.855430 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG keystone.notifications [None req-82909122-fe43-4334-b98f-6090e0edcb9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81475) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 20 13:48:37.856941 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: [pid: 81475|app: 0|req: 1301/2601] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 20 13:48:37 2022] DELETE /identity/v3/projects/e461387a7dbe4f3fabaab7634fe5f7c6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 20 13:48:47.529464 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:48:47.851961 dv-cinder-862136-1-iscsi devstack@keystone.service[81475]: DEBUG dbcounter [-] [81475] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81475) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 20 13:49:50.094529 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67248e-0619-4b19-b34b-79f4cddec11b admin admin] Authenticating user token {{(pid=81474) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 20 13:49:50.095919 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-0d67248e-0619-4b19-b34b-79f4cddec11b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:49:50.165024 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67248e-0619-4b19-b34b-79f4cddec11b admin admin] Validating token access rules against request {{(pid=81474) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 20 13:49:50.222381 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee30b73-4d05-49a0-b888-37755d54abf7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2440594934b64621a01c441133778155', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4a81b6ecfb6e46a8b08c8bc2b4000fc7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81474) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 20 13:49:50.222984 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee30b73-4d05-49a0-b888-37755d54abf7 service placement] REQUEST_METHOD: `GET` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 20 13:49:50.223266 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee30b73-4d05-49a0-b888-37755d54abf7 service placement] SCRIPT_NAME: `/identity` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 20 13:49:50.223539 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee30b73-4d05-49a0-b888-37755d54abf7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81474) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 20 13:49:50.225338 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.fernet_utils [None req-cee30b73-4d05-49a0-b888-37755d54abf7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81474) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 20 13:49:50.236227 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee30b73-4d05-49a0-b888-37755d54abf7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 20 13:49:50.237824 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee30b73-4d05-49a0-b888-37755d54abf7 service placement] RBAC: Authorization granted {{(pid=81474) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 20 13:49:50.247880 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: [pid: 81474|app: 0|req: 1301/2602] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 20 13:49:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 20 13:50:00.243670 dv-cinder-862136-1-iscsi devstack@keystone.service[81474]: DEBUG dbcounter [-] [81474] Writing DB stats keystone:SELECT=58 {{(pid=81474) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}